Display terminal, display method, and recording medium

ABSTRACT

A display terminal, a displaying method, and a program stored in a recording medium, each of which: acquires a wide-view image captured by an image capturing apparatus and having a wide angle of view; displays, on a display, a predetermined-area image representing a predetermined area of the wide-view image; and transmits an image capturing parameter to be received by the image capturing apparatus, the image capturing parameter being determined such that the quality of the predetermined-area image being displayed is adequate.

CROSS-REFERENCE TO RELATED APPLICATIONS

This patent application is based on and claims priority pursuant to 35U.S.C. §119(a) to Japanese Patent Application No. 2022-043090. filed onMar. 17, 2022. in the Japan Patent Office, the entire disclosure ofwhich is hereby incorporated by reference herein.

BACKGROUND Technical Field

The present disclosure relates to a display terminal, a display method,and a recording medium.

Related Art

A wide-field-of-view image having a wide viewing angle and captured in awide imaging range as an imaging range including even an area that isdifficult for a normal angle of view to cover is known. Thewide-field-of-view image is hereinafter referred to as a “wide-viewimage”. Examples of the wide-view image include a 360-degree image thatis a captured image of an entire 360-degree view. The 360-degree imageis also referred to as a spherical image, an omnidirectional image, oran “all-around” image. The wide-view image is generated by capturing animage of an object or surroundings such as scenery with a sphericalimaging camera.

The wide-view image looks entirely distorted to a user who views thewide-view image. A predetermined area, which is a portion of thewide-view image, may be displayed on a display for viewing.

SUMMARY

Example embodiments include a display terminal, a displaying method, anda program stored in a recording medium, each of which: acquires awide-view image captured by an image capturing apparatus and having awide angle of view; displays, on a display, a predetermined-area imagerepresenting a predetermined area of the wide-view image; and transmitsan image capturing parameter to be received by the image capturingapparatus, the image capturing parameter being determined such that thequality of the predetermined-area image being displayed is adequate.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete appreciation of embodiments of the present disclosureand many of the attendant advantages and features thereof can be readilyobtained and understood from the following detailed description withreference to the accompanying drawings, wherein:

FIG. 1 is a diagram illustrating an example of remote communicationusing a wide-view image;

FIG. 2 is a diagram illustrating an example schematic configuration of acommunication system;

FIG. 3 is a diagram illustrating an example hardware configuration of animage capturing apparatus;

FIG. 4 is a diagram illustrating an example hardware configuration of acommunication terminal and an information processing system;

FIGS. 5A, 5B, and 5C are a left side view, a front view, and a plan viewof the image capturing apparatus according to an embodiment of thepresent disclosure, respectively;

FIG. 6 is an illustration for explaining how a user uses the imagecapturing apparatus, according to an embodiment of the presentdisclosure;

FIGS. 7A, 7B, and 7C are views illustrating a hemispherical image (frontside) captured by the image capturing apparatus, a hemispherical image(back side) captured by the image capturing apparatus, and an image inequirectangular projection, respectively, according to an embodiment ofthe present disclosure:

FIG. 8A is a conceptual diagram illustrating an example of how the imagein equirectangular projection is mapped to a surface of a sphere;

FIG. 8B is a view illustrating a spherical image, according to anembodiment of the present disclosure:

FIG. 9 is a view illustrating positions of a virtual camera and apredetermined area in a case where the spherical image is of athree-dimensional sphere according to an embodiment of the presentdisclosure;

FIG. 10A is a perspective view of the virtual camera and thepredetermined area illustrated in FIG. 9 according to an embodiment ofthe present disclosure;

FIG. 10B is a view illustrating a predetermined-area image obtained inthe state illustrated in FIG. 10A and displayed on a display accordingto an embodiment of the present disclosure:

FIG. 10C is a view of a predetermined area obtained by changing thepoint of view of the virtual camera illustrated in FIG. 10A according toan embodiment of the present disclosure;

FIG. 10D is a view illustrating a predetermined-area image obtained inthe state illustrated in FIG. 10C and displayed on the display accordingto an embodiment of the present disclosure;

FIG. 11 is a view illustrating a relationship between predetermined-areainformation and an image of the predetermined area according to anembodiment of the present disclosure;

FIG. 12 is a view illustrating a point in a three-dimensional Euclideanspace defined in spherical coordinates, according to an embodiment ofthe present disclosure;

FIG. 13 is a diagram illustrating an example functional configuration ofthe communication system;

FIGS. 14A and 14B are tables illustrating an example of image managementinformation stored in an image management information storage unit;

FIGS. 15A and 15B are tables illustrating an example of virtual roominformation stored in a virtual room information storage unit and anexample of tenant information stored in a tenant information storageunit, respectively:

FIG. 16A is a view illustrating an example of a room entry screen;

FIG. 16B is a view illustrating an example of an image viewing screendisplayed on the communication terminal in response to a user entering avirtual room;

FIG. 17 is a sequence diagram illustrating an example process in whichthe user (or the communication terminal) enters the virtual room;

FIG. 18 is a view illustrating an example of a device registrationscreen displayed on the communication terminal:

FIG. 19A is a view illustrating an example of an image capturingapparatus registration dialog:

FIG. 19B is a view illustrating an example of a two-dimensional codescreen:

FIG. 20 is a view illustrating an example of a virtual reality (VR)goggles registration screen displayed in response to the pressing of aVR goggles registration button;

FIG. 21 is a view illustrating an example of a virtual room associationscreen for associating an image capturing apparatus with a virtual room;

FIG. 22 is a view illustrating an example of a virtual room associationscreen;

FIG. 23 is a view illustrating an example of a virtual room associationscreen;

FIGS. 24A and 24B are views illustrating examples of a wide-view imagetransmission control dialog displayed on the communication terminal;

FIG. 25 is a sequence diagram illustrating an example procedure in whichthe user registers the image capturing apparatus in a virtual room,

FIG. 26 is a sequence diagram illustrating an example process forsharing a wide-view image;

FIG. 27 is a view illustrating an example of an image capturing functionsetting screen:

FIG. 28 is a sequence diagram illustrating a first process for adjustingan image capturing function according to an embodiment of the presentdisclosure;

FIG. 29A is a view illustrating a wide-view image (spherical image)according to an embodiment of the present disclosure;

FIG. 29B is a view illustrating a predetermined-area image of apredetermined area in the wide-view image illustrated in FIG. 29Aaccording to an embodiment of the present disclosure:

FIG. 29C is a view illustrating a predetermined-area image obtainedafter the image capturing function is adjusted according to anembodiment of the present disclosure;

FIG. 30 is a sequence diagram illustrating a second process foradjusting an image capturing function according to an embodiment of thepresent disclosure;

FIG. 31 is a diagram illustrating an example of remote communication inwhich the communication system is used in telemedicine; and

FIG. 32 is a view illustrating an example of a virtual room associationscreen for associating an image capturing apparatus with a virtual roomfor telemedicine.

The accompanying drawings are intended to depict embodiments of thepresent disclosure and should not be interpreted to limit the scopethereof. The accompanying drawings are not to be considered as drawn toscale unless explicitly noted. Also, identical or similar referencenumerals designate identical or similar components throughout theseveral views.

DETAILED DESCRIPTION

In describing embodiments illustrated in the drawings, specificterminology is employed for the sake of clarity. However, the disclosureof this specification is not intended to be limited to the specificterminology so selected and it is to be understood that each specificelement includes all technical equivalents that have a similar function,operate in a similar manner, and achieve a similar result.

Referring now to the drawings, embodiments of the present disclosure aredescribed below. As used herein, the singular forms “a,” “an,” and “the”are intended to include the plural forms as well, unless the contextclearly indicates otherwise.

A display terminal, a system including the display terminal, and amethod performed by the display terminal and the system, according toone or more embodiments will be described hereinafter.

As described above, the wide-view image may be distorted when viewed bya user. In view of this, an image capturing apparatus may capture animage with a wide field of view while uniformly adjusting conditionssuch as exposure, white balance, and International StandardsOrganization (ISO) sensitivity over the entire wide field of view. This,however, may cause a predetermined-area image, which is a predeterminedarea in the wide-view image, to have degraded quality. In the following,the display terminal, system, and method for adjusting quality of animage of the predetermined area are described.

Example of Remote Communication

FIG. 1 is a diagram illustrating an example of remote communicationusing a wide-view image. In FIG. 1 , communication takes place acrossthree sites, namely, a site A, a site B, and a site C, via aninformation processing system 50. Three sites are merely an example, andcommunication may be performed across two sites or four or more sites.

In an example, the site A is a construction site. The sites B and C areany sites across which a wide-view image can be communicated. In anexample, the sites B and C are offices. An image capturing apparatus 10is placed at the site A. In an example, the image capturing apparatus 10can capture an image of an object and surroundings to generate awide-view image. Examples of the wide-view image include a wide-viewimage called a spherical image, and a wide-view image with a wide angleof view ranging from, for example. 180 degrees to 360 degrees in thevertical or horizontal direction. Such an image with a wide angle ofview is hereinafter simply referred to as a “wide-view image”.Communication terminals 30A to 30C for viewing a wide-view image areplaced at the sites A to C, respectively. Any communication terminal orcommunication terminals among the communication terminals 30A to 30C arehereinafter referred to as a “communication terminal 30” or“communication terminals 30”.

In the construction site, workers are involved in various constructionsat various places. An image capturing apparatus captures an image of theentire construction site to generate a wide-view image in which theentire construction site appears. A user a at the site A, a user b atthe site B. and a user c at the site C can check any construction orwork of interest to be viewed by the users a to c at the sites A to C bychanging a virtual point of view as appropriate. The term “point ofview”, as used here, refers to the center position or range of apredetermined area to be displayed on a display screen such as adisplay. The predetermined area is in the entire wide-view image.

In an example, the image capturing apparatus 10 is attached to a tripod86. In another example, the image capturing apparatus 10 is attached toan arm 85 through a gimbal 87. A relay device is installed at theconstruction site. In FIG. 1 , the communication terminal 30A alsofunctions as the relay device. The communication terminal 30A receives awide-view image from the image capturing apparatus 10 via a wire orwirelessly and transmits the received wide-view image to the informationprocessing system 50. The communication terminal 30A may also functionas a terminal for viewing the wide-view image. In an example, a camera 9connected to (or incorporated in) the communication terminal 30Acaptures an image having a normal angle of view (or a spherical image),and the captured image is transmitted to the information processingsystem 50. In another example, smart glasses 88 worn by the user agenerates an image having a normal angle of view (or a spherical image)by imaging an object, and the generated image is transmitted to theinformation processing system 50. The user a may be a worker. The smartglasses 88 are an information terminal having a display on whichinformation acquired via the Internet is displayed with a field of viewmaintained. The smart glasses 88 may be placed at any site.

The communication terminal 30B, such as a personal computer (PC) or asmartphone, is placed at the site B. The communication terminal 30B isany device that can communicate with the information processing system50. Other examples of the communication terminal 30B include a tabletterminal, a personal digital assistant (PDA), an electronic whiteboard,and a projector. A camera may be incorporated in or connected to thecommunication terminal 30B.

The communication terminal 30C, such as a PC, a smartphone, or virtualreality (VR) goggles 89, is placed at the site C. In FIG. 1 , a camera 8is incorporated in or connected to the communication terminal 30C. TheVR goggles 89 are an information terminal for displaying acomputer-based artificial world or a spherical image in accordance withthe direction of movement of the neck or the body of the user wearingthe VR goggles 89. The VR goggles 89 may be goggles attached to asmartphone for VR application. Examples of such goggles includeHACOSCO®, which is a VR scope that allows a user to easily enjoy VR byplugging a smartphone into an assembled cardboard headset with a plasticlens. The camera 8 may be for a wide angle of view or a normal angle ofview. The communication terminal 30C is any device that can communicatewith the information processing system 50. Other examples of thecommunication terminal 30C include a tablet terminal, a PDA, anelectronic whiteboard, and a projector. The VR goggles 89 may be placedin any site.

In this embodiment, communication between the image capturing apparatus10 and the communication terminals 30 is managed using a communicationgroup called a virtual room. The image capturing apparatus 10 isassociated with the virtual room. Each of the communication terminals 30(the user who operates each of the communication terminals 30) entersthe virtual room and receives a wide-view image transmitted from theimage capturing apparatus 10. As a result, the user can view thewide-view image. The smart glasses 88 and the VR goggles 89 can also beassociated with the virtual room. Like the communication terminals 30,the cameras 8 and 9 also enter the virtual room.

The users a to c at the sites A to C can each change the point of viewfor the wide-view image, as desired, using the communication terminals30A to 30C, respectively. Thus, the users a to c viewing the wide-viewimage in real time are likely to view images with different points ofview. It may be difficult for the users a to c to mutually understandeach other. In this embodiment, accordingly, information on a virtualpoint of view set for the communication terminal 30 at any one of thesites is shareable by the communication terminals 30 at the other sites.An overview of the sharing of information will be described. In thefollowing description, in an example, a point of view designated by theuser b at the site B is shared by the users a and c at the sites A andC.

The communication terminals 30A to 30C share a wide-view image (anexample of a first wide-view image). The wide-view image is generated byimaging an object with the image capturing apparatus 10. In response tothe user b making a request to capture a wide-view image while viewingthe wide-view image from any point of view on the communication terminal30B, the communication terminal 30B (an example of a first communicationterminal) transmits point-of-view information and the request to theinformation processing system 50.

In response to the request, the information processing system 50designates the point-of-view information and transmits a request to theimage capturing apparatus 10 to capture a wide-view image (either stillimage or moving image).

In response to the request, the image capturing apparatus 10 captures awide-view image (an example of a second wide-view image), and stores thewide-view image (an example of a second wide-view image) and thepoint-of-view information in association with a uniform resource locator(URL) transmitted from the information processing system 50. The URL isan example of storage location information. In FIG. 1 , the URLindicates a storage location in a storage 90. The wide-view image storedin the storage 90 can be downloaded and displayed by any communicationterminal 30.

The information processing system 50 transmits the URL to thecommunication terminal 30B.

The information processing system 50 further transmits the URL to thecommunication terminals 30A and 30C (examples of a second communicationterminal), which are in the same virtual room as that associated withthe image capturing apparatus 10 and the communication terminal 30B,automatically or in response to a request from the user b.

The communication terminals 30A and 30C access the URL and receive thepoint-of-view information and the wide-view image. Each of thecommunication terminals 30A and 30C sets and displays the point of viewidentified by the point-of-view information such that point of viewmatches the center of an image field. It should be noted that the pointof view is not necessarily made to completely match the center of theimage field. In an example, the point of view may be set and displayedso as to be included in a range near the center of the image field.

The same applies when the point of view of the user a at the site A isshared by the users b and c at the sites B and C and when the point ofview of the user c at the site C is shared by the users a and b at thesites A and B.

As described above, in a communication system 1 a according to thisembodiment, even after a wide-view image is distributed, point-of-viewinformation is shared without an instruction being given to shift apoint of view for a wide-view image generated by capturing an image suchthat a predetermined area of interest at each site is displayed. Thisfacilitates understanding among users at the respective sites.

In (3), the image capturing apparatus 10 may transmit the wide-viewimage itself to the information processing system 50. In (4), theinformation processing system 50 may transmit the wide-view image to thecommunication terminals 30A to 30C.

In the example illustrated in FIG. 1 , the image capturing apparatus 10is placed at a construction site. This embodiment is also applicable toVR education, event distribution, remote customer services, telemedicineservices, and other suitable situations. In VR education, the imagecapturing apparatus 10 is placed at a site such as a study room or alaboratory. Students can view a blackboard, an instrument, a sample, anexperimental result, or the like from remote sites while changing thepoints of view as appropriate. In event distribution, the imagecapturing apparatus 10 is placed in a venue of an event to be heldon-site. Event participants such as an audience can view the details inthe venue online from remote sites while changing the points of view asappropriate. The details in the venue include images of eventperformers, event participants, and event presenters, images of objectsinvolved in the event, such as products and exhibits, images ofmaterials involved in the event, and images of the venue. The event maybe held indoor or outdoor, and examples of the venue of the eventinclude venues such as sports stadiums, concert halls, and theaters. Inremote customer services, for example, in customer services for a travelagency, the image capturing apparatus 10 is placed at each of traveldestination sites. A customer can plan their itinerary from a remotesite while changing the point of view as appropriate. In telemedicineservices, in an example, the image capturing apparatus 10 is placed in amedical setting such as an operating room. Medical people such asdoctors, medical students, and persons related to medical instrumentscan view the performance of a doctor(s) and a nurse(s) during on-sitemedical treatment, the arrangement of medical instruments, the state ofa patient, vitals, and the like from remote sites while changing thepoints of view as appropriate.

The site at which an image is captured is not limited to any of thesites described above. An image may be captured in any space that a user(or viewer) at a viewing site desires to remotely grasp. Examples ofsuch a space include a school, a factory, a warehouse, a building site,a server room, and a store.

The term “tenant” refers to a group of users associated with a unitcontract for receiving an image distribution service from a serviceprovider (information processing system in this embodiment). Examples ofthe tenant include entities that have made the contract, such as acompany, an organization, and an individual. Accordingly, a tenant mayalso be referred to as a user group. In one example, a user belongs tothe tenant. In another example, a user may personally subscribe to theservice. A user, an image capturing apparatus, a virtual room, and thelike are registered in a tenant (user group).

The term “site” refers to a location where activity takes place. In thisembodiment, a conference room is used as an example of a site. Theconference room is a room to be used mainly for a conference. Aconference is an event where people gather to discuss something and isalso referred to as a meeting, a session, a gathering, an assembly, orthe like.

The term “device” refers to an apparatus different from thecommunication terminal 30 for general purposes such as a PC or asmartphone. In an example, the device is an image capturing apparatus oran apparatus for viewing a wide-view image. In this embodiment, examplesof the device include the image capturing apparatus 10. the smartglasses 88. and the VR goggles 89.

The term “point-of-view information” refers to parameter informationthat specifies which predetermined area in a wide-view image to bedisplayed on the display screen of the display is to be displayed on thedisplay screen of the display. In this embodiment, in an example, thepoint-of-view information includes a radius vector, a polar angle, andan azimuth angle of the center of the wide-view image to be displayed onthe display screen of the display. In another example, the point-of-viewinformation may be specified by other parameter information such as thecoordinates of diagonal vertices.

The term “wide-view image” refers to an image having a viewing angle ina wider range than a display range that can be displayed on the displayscreen (area where the wide-view image is to be displayed) of thedisplay at a time in a predetermined display method. The wide-view imagehas a display range corresponding to a field of view up to 360 degrees(or 180 degrees) in the vertical direction and a field of view up to 360degrees in the horizontal direction. In an example, the wide-view imageis an image having a display range corresponding to a field of view ofless than 360 degrees in the vertical and horizontal directions as longas the wide-view image has a viewing angle in a wider range than thedisplay range that can be displayed on the display screen of the displayat a time. In another example, the wide-view image is an image having adisplay range corresponding to a field of view of 160 degrees or more inthe vertical and horizontal directions. Examples of the wide-view imageinclude an image having a display range wider than a range that can bevisually recognized at a time by a person looking at the range.Depending on the display method, an image that can be displayed on thedisplay screen of the display at a time is also the wide-view image aslong as the image has a viewing angle in a wide range in response to thedisplay method being switched to a predetermined display method orchanged. In this embodiment, a spherical image in equirectangularprojection format is used as an example of a wide-view image. Otherexamples of the wide-view image include an omnidirectional image, ahemispherical image, a three-dimensional (3D) panoramic image, atwo-dimensional (2D) panoramic image, and a VR image. The wide-viewimage may be in cube mapping format or dome master format. The sphericalimage may be in format other than equirectangular projection format.

An image captured at a normal angle of view is not a wide-view image. Inthis embodiment, such an image is referred to as a non-wide-view image,that is, a planar image.

The term “communication group” refers to a group of users who share awide-view image, that is, a group of users to whom a wide-view image isto be distributed. The communication group will be described with theterm “virtual room” in the sense that in a typical space, the users inthe same room can share a wide-view image. As used herein, the term“virtual” means being implemented by information processing via anetwork.

Users at respective sites perform remote communication across remotelocations. The remote communication is a meeting, which is an onlinemeeting, accessible from remote locations, or sites. The meeting means agathering of people for consultation, discussion, or the like. Examplesof the meeting include, but are not limited to, serving a customer, aconference, a gathering, an assembly, a study session, a class, aseminar, and a presentation. The remote communication is not necessarilybidirectional communication. Thus, the virtual room may be referred toas a virtual conference room.

Example Configuration of Communication System

FIG. 2 is a diagram illustrating an example schematic configuration ofthe communication system 1 a. In FIG. 1 , in an example, thecommunication system 1 a illustrated in FIG. 2 is applied to remotecommunication with a construction site. The communication system 1 a isa system for transmitting and receiving a wide-view image captured bythe image capturing apparatus 10 or an image having a normal angle ofview bidirectionally among a plurality of sites. In the communicationsystem 1 a, an image distributed from one of the sites is displayed atthe other sites and is viewable by users at the other sites. In anexample, a spherical image captured by the image capturing apparatus 10is distributed as the wide-view image. In the communication system 1 a,for example, a wide-view image captured at a predetermined site isremotely viewable at another site.

In the communication system 1 a, as illustrated in FIG. 2 , the imagecapturing apparatus 10 and the communication terminal 30A placed at thesite A, the information processing system 50, and the communicationterminals 30B and 30C placed at a plurality of sites, namely, the sitesB and C, respectively, are communicably connected to each other.

In a case where the image capturing apparatus 10 has a communicationfunction capable of directly connecting to a communication network N,the communication terminal 30A serving as a relay device (e.g., arouter) is not used. In this case, the image capturing apparatus 10 isconnected to the communication network N without the use of thecommunication terminal 30A. In a case where the communication terminal30A is placed at the site A, the communication terminal 30A alsofunctions as a relay device, and the user a can view a wide-view imagein a manner similar to that of the communication terminals 30B and 30C.The image capturing apparatus 10 may additionally be placed at a siteother than the site A, or a plurality of image capturing apparatuses 10may be placed at the site A.

Each communication terminal 30 and the information processing system 50can communicate with each other via the communication network N. Thecommunication network N includes the Internet, a mobile communicationnetwork, and a local area network (LAN), for example. The communicationnetwork N may include a wired communication network and a wirelesscommunication network. The wireless communication network may be basedon a wireless communication standard such as third generation (3G),fourth generation (4G), fifth generation (5G), Wireless Fidelity(Wi-Fi®), Worldwide Interoperability for Microwave Access (WiMAX), orLong Term Evolution (LTE).

The image capturing apparatus 10 is a digital camera that can capture animage of an object or surroundings such as scenery to obtain twohemispherical images, from which a spherical image is generated, asdescribed below. The wide-view image obtained by the image capturingapparatus 10 may be a moving image or a still image, or may include bothof a moving image and a still image. Further, the captured image may bevideo including audio together with an image.

The communication terminal 30 is a computer such as a PC to be operatedby a user at each site. The communication terminal 30 displays an imageobtained by imaging an object at the site where the communicationterminal 30 is placed, and a wide-view image (still image and/or movingimage) and an image having a normal angle of view, which are distributedfrom other sites. For example, the communication terminal 30 acquires awide-view image, which is captured by the image capturing apparatus 10,via the communication network N. The communication terminal 30 hasinstalled therein software for executing image processing, such as OpenGraphics Library for Embedded Systems (OpenGL ES), and can display animage based on point-of-view information that specifies a partial areain the wide-view image. OpenGL ES is an example of software forexecuting image processing. Any other software may be used. In anexample, the communication terminal 30 does not have installed thereinsoftware for executing image processing, and executes image processingby using software received from the outside or receives a result ofimage processing executed by external software to display an image. Thatis, the communication terminal 30 can display a predetermined area,which is part of the wide-view image.

The communication terminal 30 can change the point of view for thedisplay range of the wide-view image, as desired, in response to theuser’s operation. The communication terminal 30 shifts the virtual pointof view in response to a user operation input (such as key input,dragging, or scrolling) on a touch panel, a direction button, a mouse, akeyboard, a touch pad, or the like to change and display a visual fieldrange (predetermined area) based on point-of-view informationcorresponding to the shifted point of view. In an example, thecommunication terminal 30 is a communication terminal to be worn by theuser, such as VR goggles. In response to a change in the movement of theuser wearing the communication terminal 30. position information of thecommunication terminal 30 is changed. In response to detection of thechange in the position information, the virtual point of view is shiftedin accordance with the detected position information to change a visualfield range (predetermined area), based on point-of-view informationcorresponding to the shifted point of view, and the changed visual fieldrange (predetermined area) is displayed.

The communication terminal 30A acquires a wide-view image from the imagecapturing apparatus 10 via a wired cable such as a Universal Serial Bus(USB) cable connected to an input/output interface (I/F) 116 describedbelow. The communication terminal 30A distributes the acquired wide-viewimage to the communication terminal 30 at another site via theinformation processing system 50. The connection between the imagecapturing apparatus 10 and the communication terminal 30A may be eithera wired connection using a wired cable or a wireless connection usingshort-range wireless communication, for example. A plurality ofcommunication terminals 30A may be placed at the site A.

In an example, the user a at the site A wears the smart glasses 88, andthe smart glasses 88 are connected to the communication network N. Animage captured by the smart glasses 88 is transmitted to the informationprocessing system 50 via the communication network N, and theinformation processing system 50 can distribute the image to thecommunication terminal 30 at each site.

The communication terminal 30B is placed at the site B where the user bis located, and the communication terminal 30C is placed at the site Cwhere the user c is located. A plurality of communication terminals 30Bmay be placed at the site B, and a plurality of communication terminals30C may be placed at the site C. The users b and c may carry thecommunication terminals 30B and 30C, respectively.

Each of the communication terminals 30A to 30C at the sites A to C canbe internally or externally provided with the camera 8 or 9. The cameras8 and 9 are examples of an imaging device. Each of the communicationterminals 30A to 30C can distribute an image of the corresponding one ofthe sites A to C, which is captured by the camera 8 or 9 thereof, to theother sites. Any device may be placed at each of the sites A to C.

The arrangement of the terminals and apparatus (i.e., the communicationterminals 30 and the image capturing apparatus 10) and the users a to cillustrated in FIG. 2 is an example. Any other arrangement may be used.The communication terminal 30 is not limited to a PC and may be, forexample, a tablet terminal, a smartphone, a PDA, a wearable terminal(including smart glasses or VR goggles), a projector (PJ), anInteractive White Board (IWB), which is an electronic whiteboard withmutual communication capability, a telepresence robot, or the like. Thecommunication terminal 30 is any computer on which a web browser or anapplication dedicated to an image distribution service operates.

In an example, the image capturing apparatus 10 includes a display anddisplays an image distributed from another site on the display.

The information processing system 50 includes one or more informationprocessing apparatuses. The information processing system 50 manages andcontrols communication among the image capturing apparatus 10 and thecommunication terminals 30 at the respective sites and manages awide-view image to be transmitted and received. The informationprocessing system 50 provides a platform on which a function ofproviding an image distribution service for distributing a wide-viewimage is available. The platform may be made available to a person, acompany, or any other service provider that desires to provide an imagedistribution service, under contract. A service provider that providesan image distribution service to a user by using a contracted platformis hereinafter referred to as a platform contractor to distinguish theservice provider from a tenant who receives the image distributionservice.

The information processing system 50 may publish an applicationprogramming interface (API) as a platform, and the platform contractormay use the API to provide various image distribution services. Theplatform contractor mainly develops software such as an application forcalling the API or the screen to be displayed on the communicationterminal 30. That is, the functions to be provided by the API, such asimage distribution, do not have to be developed from scratch.

The information processing system 50 may be implemented by a singlecomputer or a plurality of computers such that the components (functionsor means) of the information processing system 50 are divided into andassigned to the plurality of computers as appropriate. All or some ofthe functions of the information processing system 50 may be implementedby a server computer residing in a cloud environment or a servercomputer residing in an on-premise environment.

The storage 90 is a storage device that stores data such as a wide-viewimage. In an example, the storage 90 is an external storage separatefrom the information processing system 50. The external storage may be acloud or on-premise storage. In another example, the storage 90 is astorage included in the information processing system 50.

Example Hardware Configuration

Next, the hardware configurations of each apparatus or terminal includedin the communication system 1 a according to this embodiment will bedescribed with reference to FIGS. 3 and 4 . In the hardwareconfigurations illustrated in FIGS. 3 and 4 , certain hardware elementsmay be added or deleted as appropriate.

Hardware Configuration of Image Capturing Apparatus

First, the hardware configuration of the image capturing apparatus 10will be described with reference to FIG. 3 . FIG. 3 is a diagramillustrating an example hardware configuration of the image capturingapparatus 10. In the following description, the image capturingapparatus 10 is a spherical (omnidirectional) image capturing apparatusincluding two imaging elements. However, the image capturing apparatus10 may include one imaging element or three or more imaging elements. Inone example, the image capturing apparatus 10 is not dedicated toomnidirectional image capturing, and an external omnidirectional imagecapturing unit is attached to a general-purpose digital camera or asmartphone to implement functions that are substantially the same asthose of the image capturing apparatus 10.

As illustrated in FIG. 3 , the image capturing apparatus 10 includes animaging unit 101, an image processor 104, an imaging controller 105, amicrophone 108, an audio processor 109, a central processing unit (CPU)111, a read only memory (ROM) 112, a static random access memory (SRAM)113, a dynamic random access memory (DRAM) 114, an operation unit 115.an input/output interface (I/F) 116. a short-range communication circuit117, an antenna 117 a for the short-range communication circuit 117, anelectronic compass 118, a gyro sensor 119, an acceleration sensor 120,and a network I/F 121.

The imaging unit 101 includes two wide-angle lens covers 102 a and 102 b(collectively referred to as lens covers 102 unless distinguished) andtwo imaging elements 103 a and 103 b corresponding to the lens covers102 a and 102 b respectively. The configuration of the imaging unit 101captures images with an angle of view of equal to or greater than 180degrees to form two hemispherical images. The imaging unit 101 furtherincludes lens units 106 a and 106 b and mechanical shutters 107 a and107 b. The lens units 106 a and 106 b and the mechanical shutters 107 aand 107 b are disposed more inward than the lens covers 102 a and 102 b.respectively, and arranged in this order from the lens covers 102 a and102 b to the imaging elements 103 a and 103 b, respectively. The imagingunit 101 having the configuration described above can adjust the shutterspeed, the focus, and the like. In an example, the imaging unit 101 mayinclude electronic shutters in place of the mechanical shutters 107 aand 107 b.

Each of the imaging elements 103 a and 103 b includes an image sensorsuch as a complementary metal oxide semiconductor (CMOS) sensor or acharge-coupled device (CCD) sensor, a timing generation circuit, and agroup of registers. The image sensor converts an optical image formed bythe lens unit 106 a or 106 b and the like into an electric signal andoutputs image data. The timing generation circuit generates horizontalor vertical synchronization signals, pixel clocks, and the like for theimage sensor. In the group of registers, various commands, parameters,and the like for an operation of the imaging element 103 a or 103 b areset. As a non-limiting example, the imaging unit 101 includes twowide-angle lenses. The imaging unit 101 may include one wide-angle lensor three or more wide-angle lenses.

Each of the imaging elements 103 a and 103 b of the imaging unit 101 isconnected to the image processor 104 via a parallel I/F bus. Further,each of the imaging elements 103 a and 103 b of the imaging unit 101 isconnected to the imaging controller 105 via a serial I/F bus such as aninter-integrated circuit (12C) bus.

The image processor 104, the imaging controller 105, and the audioprocessor 109 are connected to the CPU 111 via a bus 110. The ROM 112,the SRAM 113, the DRAM 114, the operation unit 115. the input/output I/F116, the short-range communication circuit 117, the electronic compass118, the gyro sensor 119, the acceleration sensor 120, and the networkI/F 121 are also connected to the bus 110.

The image processor 104 acquires respective items of image data outputfrom the imaging elements 103 a and 103 b via the parallel I/F buses andperforms predetermined processing on the items of image data.Thereafter, the image processor 104 combines the items of image data togenerate data of an equirectangular projection image (an example of awide-view image) described below.

The imaging controller 105 usually functions as a master device whileeach of the imaging elements 103 a and 103 b usually functions as aslave device. The imaging controller 105 sets commands and the like inthe group of registers of each of the imaging elements 103 a and 103 bvia the 12C bus. The imaging controller 105 receives various commandsfrom the CPU 111. Further, the imaging controller 105 controls drivingof the lens units 106 a and 106 b and the mechanical shutters 107 a and107 b (electronic shutters). Further, the imaging controller 105acquires status data and the like of the group of registers of each ofthe imaging elements 103 a and 103 b via the 12C bus. The imagingcontroller 105 sends the obtained status data and the like to the CPU111.

The imaging controller 105 instructs the imaging elements 103 a and 103b to output the image data at the time when a shutter button of theoperation unit 115 is pressed. In one example, the image capturingapparatus 10 displays a preview image or a moving image (movie) on adisplay. Examples of the display include a display of a smartphone orany other external terminal that performs short-range communication withthe image capturing apparatus 10 through the short-range communicationcircuit 117. In the case of displaying movie, image data arecontinuously output from the imaging elements 103 a and 103 b at apredetermined frame rate (frames per minute).

The imaging controller 105 operates in cooperation with the CPU 111 tosynchronize the time when the imaging element 103 a outputs image dataand the time when the imaging element 103 b outputs the image data. Inthis embodiment, the image capturing apparatus 10 does not include adisplay unit (or display). In some embodiments, the image capturingapparatus 10 may include a display unit. The microphone 108 convertssound to audio data (signal). The audio processor 109 acquires the audiodata output from the microphone 108 via an I/F bus and performspredetermined processing on the audio data.

The CPU 111 controls entire operation of the image capturing apparatus10 and performs predetermined processing. The ROM 112 stores variousprograms for execution by the CPU 111. Each of the SRAM 113 and the DRAM114 operates as a work memory to store programs to be executed by theCPU 111 or data being currently processed. More specifically, in oneexample, the DRAM 114 stores image data currently processed by the imageprocessor 104 and data of the equirectangular projection image on whichprocessing has been performed.

The operation unit 115 collectively refers to various operation keys, apower switch, a shutter button, a touch panel having both the displayand operation functions, and the like. The user operates the operationunit 115 to input various image capturing modes or image capturingconditions.

The input/output I/F 116 collectively refers to an interface circuitsuch as a USB I/F that allows the image capturing apparatus 10 tocommunicate with an external medium such as a Secure Digital (SD) cardor an external personal computer. The input/output I/F 116 may be eitherwired or wireless. The data of the equirectangular projection image,which is stored in the DRAM 114, is stored in the external medium viathe input/output I/F 116 or transmitted to an external terminal(apparatus) via the input/output I/F 116, as desired.

The short-range communication circuit 117 communicates with the externalterminal (apparatus) via the antenna 117 a of the image capturingapparatus 10 by short-range wireless communication technology such asnear field communication (NFC), Bluetooth®, or Wi-Fi®. The short-rangecommunication circuit 117 can transmit the data of the equirectangularprojection image to the external terminal (apparatus).

The electronic compass 118 calculates an orientation of the imagecapturing apparatus 10 from the Earth’s magnetism and outputsorientation information. The orientation information is an example ofrelated information (metadata) in compliance with exchangeable imagefile format (Exif). The orientation information is used for imageprocessing such as image correction of a captured image. The relatedinformation also includes data of a date and time when the image wascaptured, and data of a data size of image data.

The gyro sensor 119 detects a change in tilt (roll, pitch, and yaw) ofthe image capturing apparatus 10 with movement of the image capturingapparatus 10. The change in tilt is one example of related information(metadata) in compliance with Exif. This information is used for imageprocessing such as image correction of a captured image.

The acceleration sensor 120 detects acceleration in three axialdirections. The image capturing apparatus 10 calculates the position ofthe image capturing apparatus 10 (e.g., the tilt of the image capturingapparatus 10 relative to the direction of gravity), based on theacceleration detected by the acceleration sensor 120. The gyro sensor119 and the acceleration sensor 120 of the image capturing apparatus 10improve the accuracy of image correction.

The network I/F 121 is an interface for performing data communicationusing the communication network N, such as the Internet, via a router orthe like. The hardware elements of the image capturing apparatus 10 arenot limited to the illustrated ones as long as the functionalconfiguration of the image capturing apparatus 10 can be implemented. Atleast some of the hardware elements described above may reside on thecommunication network N. Hardware Configuration of CommunicationTerminal

FIG. 4 is a diagram illustrating an example hardware configuration ofthe communication terminal 30 and the information processing system 50.First, the communication terminal 30 will be described. Each hardwareelement of the communication terminal 30 is denoted by a referencenumeral in 300 series. The communication terminal 30 is implemented byone or more computers. As illustrated in FIG. 4 , the communicationterminal 30 includes a CPU 301, a ROM 302. a RAM 303, a hard disk drive(HDD) 304, an HDD controller 305, a display 306, an external deviceconnection I/F 308, a network I/F 309, a bus line 310, a keyboard 311, apointing device 312, a digital versatile disc rewritable (DVD-RW) drive314, a media I/F 316, an audio input/output I/F 317. a microphone 318, aspeaker 319, a short-range communication circuit 320, and a camera 321.

The CPU 301 controls entire operation of the communication terminal 30.The ROM 302 stores a control program such as an initial program loader(IPL) for driving the CPU 301. The RAM 303 is used as a work area forthe CPU 301. The HDD 304 stores a program and various data. The HDDcontroller 305 controls reading or writing of various data from or tothe HDD 304 under control of the CPU 301. The display 306 displaysvarious kinds of information such as a cursor, a menu, a window,characters, and an image. The display 306 is an example of a displayunit. In one example, the display 306 is a touch panel display providedwith an input means. The external device connection I/F 308 is aninterface for connecting to various external devices. The externaldevices include, but are not limited to, a USB memory and a printer. Thenetwork I/F 309 is an interface for performing data communication usingthe communication network N. The bus line 310 is an address bus or adata bus for electrically connecting the hardware elements illustratedin FIG. 4 , such as the CPU 301, to each other. The HDD 304 and the HDDcontroller 305 are each an example of a storage that stores a program,data, and the like, and may be a solid state drive (SSD) and an SSDcontroller, respectively.

The keyboard 311 is an example of an input means including a pluralityof keys for inputting characters, numerical values, variousinstructions, and the like. The pointing device 312 is an example of aninput means used for selecting or executing various instructions,selecting a target for processing, or moving a cursor being displayed.The input means are not limited to the keyboard 311 and the pointingdevice 312 and may be a touch panel, a voice input device, or the like.The DVD-RW drive 314 controls reading or writing of various data from orto a DVD-RW 313, which is an example of a removable recording medium. Adigital versatile disc recordable (DVD-R) disc, a Blu-ray Disc®, or anyother recording medium may be used instead of the DVD-RW 313. The mediaI/F 316 controls reading or writing (storing) of data from or to arecording medium 315 such as a flash memory. The microphone 318 is anexample of a built-in sound collecting means for receiving input sounds.The audio input/output I/F 317 is a circuit for controlling input andoutput of audio signals between the microphone 318 and the speaker 319under control of the CPU 301. The short-range communication circuit 320communicates with the external terminal (apparatus) by short-rangewireless communication technology such as NFC. Bluetooth®, or Wi-Fi®.The camera 321 is an example of a built-in image capturing means forcapturing an image of an object to obtain image data. In one example,the microphone 318, the speaker 319, and the camera 321 are devicesexternal to the communication terminal 30 in alternative to built-indevices.

The hardware elements of the communication terminal 30 are not limitedto the illustrated ones as long as the functional configuration of thecommunication terminal 30 can be implemented. At least some of thehardware elements described above may reside on the communicationnetwork N.

Hardware Configuration of Information Processing System

As illustrated in FIG. 4 , each hardware element of the informationprocessing system 50 is denoted by a reference numeral in 500 series inparentheses. The information processing system 50 is implemented by oneor more computers and has substantially the same configuration as thatof the communication terminal 30 illustrated in FIG. 4 , and thus thedescription of the hardware configuration of the information processingsystem 50 will be omitted.

The hardware elements of the information processing system 50 are notlimited to the illustrated ones as long as the functional configurationof the information processing system 50 can be implemented. At leastsome of the hardware elements described above may reside on thecommunication network N.

Each of the programs described above may be recorded as a file in aformat installable or executable on a computer-readable recording mediumfor distribution. Examples of the recording medium include a compactdisc recordable (CD-R), a digital versatile disc (DVD), a Blu-ray Disc®,an SD card, and a USB memory. The recording medium may be provided inthe form of a program product to users within a certain country oroutside that country. For example, in the communication terminal 30, aprogram according to an embodiment of the present disclosure is executedto implement an image display method according to an embodiment of thepresent disclosure.

Wide-View Image and Point-of-View Information

A method for generating a wide-view image (spherical image) will bedescribed hereinafter with reference to FIGS. 5A to 12 .

First, the external appearance of the image capturing apparatus 10 willbe described with reference to FIGS. 5A to 5C. The image capturingapparatus 10 is a digital camera for capturing images from which a360-degree spherical image is generated. FIG. 5A is a left side view ofthe image capturing apparatus 10. FIG. 5B is a front view of the imagecapturing apparatus 10. FIG. 5C is a plan view of the image capturingapparatus 10. The illustrated external view of the image capturingapparatus 10 is merely an example. The image capturing apparatus 10 mayhave any other external appearance.

As illustrated in FIG. 5A, the image capturing apparatus 10 has a sizesuch that a person can hold the image capturing apparatus 10 with onehand. The illustrated shape of the image capturing apparatus 10 is anexample. The image capturing apparatus 10 may have any other shape. Asillustrated in FIGS. 5A, 5B, and 5C, the imaging element 103 a and theimaging element 103 b are disposed in an upper portion of the imagecapturing apparatus 10 such that the imaging element 103 a is disposedon the front side and the imaging element 103 b is disposed on the backside. The imaging elements (image sensors) 103 a and 103 b are used incombination with optical members (e.g., the lens units 106 a and 106 bdescribed above), each being configured to capture a hemispherical imagehaving an angle of view of equal to or greater than 180 degrees. Asillustrated in FIG. 5B, the operation unit 115, such as a shutterbutton, is disposed on the back surface of the image capturing apparatus10. As described above, the image capturing apparatus 10 may include oneimaging element or three or more imaging elements.

Next, a situation where the image capturing apparatus 10 is used will bedescribed with reference to FIG. 6 . FIG. 6 is an illustration forexplaining how a user uses the image capturing apparatus 10. Asillustrated in FIG. 6 , for example, the image capturing apparatus 10 isused for capturing an image of objects surrounding the image capturingapparatus 10. Each of the imaging elements 103 a and 103 b illustratedin FIGS. 5A to 5C captures an image of the objects surrounding the imagecapturing apparatus 10. As a result, two hemispherical images areobtained.

Next, an overview of a process for generating a spherical image from theimages captured by the image capturing apparatus 10 will be describedwith reference to FIGS. 7A to 7C and FIGS. 8A and 8B. FIG. 7A is a viewillustrating a hemispherical image (front side) captured by the imagecapturing apparatus 10. FIG. 7B is a view illustrating a hemisphericalimage (back side) captured by the image capturing apparatus 10. FIG. 7Cis a view illustrating an image in equirectangular projection(hereinafter referred to as an “equirectangular projection image” or an“equidistant cylindrical projection image”). FIG. 8A is a conceptualdiagram illustrating how the equirectangular projection image is mappedto a surface of a sphere. FIG. 8B is a view illustrating a sphericalimage. The term “equirectangular projection image” refers to a sphericalimage in equirectangular projection format, which is an example of thewide-view image described above.

As illustrated in FIG. 7A, an image obtained by the imaging element 103a is a curved hemispherical image (front side) captured through the lensunit 106 a described above. As illustrated in FIG. 7B, an image capturedby the imaging element 103 b is a curved hemispherical image (back side)captured through the lens unit 106 b described above. The imagecapturing apparatus 10 combines the hemispherical image (front side) andthe hemispherical image (back side), which are flipped by 180 degrees,to create an equirectangular projection image EC as illustrated in FIG.7C.

The image capturing apparatus 10 uses software such as OpenGL ES to mapthe equirectangular projection image EC so as to cover a surface of asphere as illustrated in FIG. 8A, to generate a spherical image (orspherical panoramic image) CE as illustrated in FIG. 8B. That is, thespherical image CE is represented as the equirectangular projectionimage EC, which corresponds to a surface facing the center of thesphere. OpenGL ES is a graphics library used for visualizing 2D and 3Ddata. OpenGL ES is an example of software for executing imageprocessing. Any other software may be used to create the spherical imageCE. The spherical image CE may be either a still image or a movingimage. As a non-limiting example, the image capturing apparatus 10generates a spherical image. In another example, the informationprocessing system 50 or the communication terminal 30 executes similarimage processing or some steps of the image processing.

Since the spherical image CE is an image mapped so as to cover thesurface of the sphere, part of the image may look distorted when viewedby a user, providing a strange feeling. To overcome the strange feeling,the image capturing apparatus 10 or the communication terminal 30displays an image of a predetermined area T, which is part of thespherical image CE. as a flat image having fewer curves to make the userfeel comfortable when viewing the image. The image of the predeterminedarea T is referred to as a “predetermined-area image”. The display ofthe predetermined-area image will be described with reference to FIGS. 9and 10A to 10D.

FIG. 9 is a view illustrating the position of a virtual camera IC andthe position of the predetermined area T in a case where the sphericalimage CE is of a three-dimensional sphere CS. The position of thevirtual camera IC corresponds to the position of a virtual point of viewof a user who is viewing the spherical image CE represented as a surfacearea of the three-dimensional sphere CS. FIG. 10A is a perspective viewof the virtual camera IC and the predetermined area T illustrated inFIG. 9 . FIG. 10B is a view illustrating a predetermined-area imageobtained in the state illustrated in FIG. 10A and displayed on adisplay. FIG. 10C is a view of a predetermined area obtained by changingthe point of view of the virtual camera IC illustrated in FIG. 10A. FIG.10D is a view illustrating a predetermined-area image obtained in thestate illustrated in FIG. 10C and displayed on the display.

Assuming that the spherical image CE generated in the way describedabove is a surface area of the sphere CS, the virtual camera IC isinside the spherical image CE as illustrated in FIG. 9 . Thepredetermined area T in the spherical image CE is an imaging area of thevirtual camera IC. Specifically, the predetermined area T is specifiedby predetermined-area information indicating an imaging direction and anangle of view of the virtual camera IC in a three-dimensional virtualspace containing the spherical image CE. Zooming in or out of thepredetermined area T can be implemented by bringing the virtual cameraIC closer to or farther away from the spherical image CE. Apredetermined-area image Q is the image of the predetermined area T inthe spherical image CE. The predetermined area T is defined by an angleof view α of the virtual camera IC and a distance f from the virtualcamera IC to the spherical image CE (see FIG. 11 ).

The predetermined-area image Q illustrated in FIG. 10A is displayed on apredetermined display as an image of the imaging area of the virtualcamera IC, as illustrated in FIG. 10B. The image illustrated in FIG. 10Bis a predetermined-area image represented by predetermined-areainformation that is set by default. A description will be made using theimaging direction (ea, aa) and the angle of view (α) of the virtualcamera IC. In another example, the predetermined area T is not definedby the angle of view α and the distance f, and the imaging area of thevirtual camera IC, which is the predetermined area T. is identified byposition coordinates (X, Y, Z).

As illustrated in FIG. 10C, in response to the movement (also referredto as “shift”) of the point of view of the virtual camera IC to theright (i.e.. to the left from the viewer’s perspective) from the stateillustrated in FIG. 10A, the predetermined area T in the spherical imageCE is moved to a predetermined area T′. Accordingly, thepredetermined-area image Q displayed on the predetermined display ischanged to a predetermined-area image Q′. As a result, the displaydisplays the image illustrated in FIG. 10D instead of the imageillustrated in FIG. 10B.

Next, a relationship between the predetermined-area information and theimage of the predetermined area T will be described with reference toFIG. 11 . FIG. 11 is a view illustrating a relationship between thepredetermined-area information and the image of the predetermined areaT. As illustrated in FIG. 11 , “ea” denotes an elevation angle, “aa”denotes an azimuth angle, and “α” denotes an angle of view of thevirtual camera IC. The position of the virtual camera IC is adjustedsuch that the point of gaze of the virtual camera IC. indicated by theimaging direction (ea, aa), matches a center point CP (x, y) of thepredetermined area T serving as the imaging area of the virtual cameraIC. As illustrated in FIG. 11 , the center point CP (x, y) of thepredetermined area T, whose diagonal angle of view is represented by theangle of view α of the virtual camera IC and is denoted by α, is used asa parameter (x, y) of the predetermined-area information. Thepredetermined-area image Q is the image of the predetermined area T inthe spherical image CE. The distance f is a distance from the virtualcamera IC to the center point CP (x, y) of the predetermined area T. Thedistance between the center point CP (x, y) and a given vertex of thepredetermined area T is denoted by “L” (2L is a diagonal line). In FIG.11 , a trigonometric function generally expressed by Equation (1) belowholds.

$\begin{matrix}{{\text{L}/\text{f}} = \tan\left( \frac{\alpha}{2} \right)} & \text{­­­(1)}\end{matrix}$

The image capturing apparatus 10 described above is an example of animage capturing apparatus that can acquire a wide-view image. Thespherical image CE is an example of a wide-view image. The wide-viewimage is generally an image taken with a wide-angle lens such as a lenscapable of taking a range wider than a range that the human eye canperceive.

FIG. 12 is a view illustrating the relationship illustrated in FIG. 11using a point in a three-dimensional Euclidean space defined inspherical coordinates. The center point CP illustrated in FIG. 11 isrepresented by a spherical polar coordinate system to obtain positioncoordinates (r, θ. φ. The position coordinates (r, θ, φ) represent aradius vector, a polar angle, and an azimuth angle. The radius vector ris a distance from the origin of the three-dimensional virtual spaceincluding the spherical image CE to the center point CP. Accordingly,the radius vector r is equal to the distance f illustrated in FIG. 11 .FIG. 12 illustrates the relationship illustrated in FIG. 11 . In thefollowing description, the position coordinates (r, θ, φ) of the virtualcamera IC are used as an example of point-of- view information. Asdescribed above, the point-of-view information is any parameterinformation that can define the predetermined area T (thepredetermined-area image Q) displayed on the predetermined displayillustrated in FIG. 10A as the image of the imaging area of the virtualcamera IC. The point-of-view information includes the coordinates of thediagonal vertices of the predetermined area T. In an example, thepoint-of-view information includes information indicating the angle ofview α of the virtual camera IC and information indicating the centerpoint CP (x, y), which have been described with reference to FIG. 11 .In another example, the point-of-view information includes informationindicating the angle of view α of the virtual camera IC and informationindicating the azimuth angle aa, which have been described withreference to FIG. 11 . Examples of the point-of-view information includeposition coordinate information in the form of spherical coordinates,position coordinate information in the form of orthogonal coordinates,and a difference value between the predetermined-area information thatis set by default and the coordinates. Other examples of thepoint-of-view information include information other than coordinateinformation, such as an angle and a distance, as illustrated in FIG. 11. In FIGS. 11 and 12 , the center point CP of the predetermined area Tis used as a reference. In another example, the predetermined area T maybe defined by parameter information with any one of the vertices of thepredetermined area T as a reference. In the foregoing description of thepoint-of-view information, as a non-limiting example, the wide-viewimage is a spherical image. In any other wide-view image, informationthat defines the predetermined area T in the other wide-view image ispoint-of-view information.

The point-of-view information may include parameter information such asthe height and width of the predetermined area T or parameterinformation such as the magnification factor of the virtual camera IC byzooming in or out. In an example, the point-of-view information isparameter information such as the direction and the angle of view of thevirtual camera IC in a case where the position of each pixel in theequirectangular projection image EC illustrated in FIG. 7C is associatedwith the coordinates of the corresponding position on the surface of thesphere (e.g., coordinates on two axes, namely, latitude and longitude).In another example, the point-of-view information includes informationsuch as latitude and longitude. As described above, the point-of-viewinformation is not limited to information indicating a point.

Functions

Next, the functional configuration of the communication system 1 aaccording to this embodiment will be described with reference to FIG. 13. FIG. 13 is a diagram illustrating an example functional configurationof the communication system 1 a according to this embodiment. FIG. 13illustrates functions, related to processes or operations describedbelow, of the terminals, the apparatus, and the server illustrated inFIG. 1 .

Functional Configuration of Image Capturing Apparatus

First, the functional configuration of the image capturing apparatus 10will be described with reference to FIG. 13 . The image capturingapparatus 10 includes a communication unit 11, an acceptance unit 12, animaging processing unit 13, an analysis unit 14, a registration requestunit 15, a connection unit 16, a storage processing unit 17, an imagetransmission control unit 18, a storing/reading unit 19, and anadjustment unit 21. Each of these units is a function or means that isimplemented by any one of the hardware elements illustrated in FIG. 3operating in accordance with instructions from the CPU 111 according tothe program loaded onto the SRAM 113 or the DRAM 114. The imagecapturing apparatus 10 further includes a storage unit 1000. The storageunit 1000 is implemented by the ROM 112 and the like illustrated in FIG.3 .

The communication unit 11 is mainly implemented by the short-rangecommunication circuit 117 operating in accordance with instructions fromthe CPU 111. The communication unit 11 is a function of connecting tothe communication network N by using wireless communication technologysuch as Wi-Fi® to transmit and receive various data or information toand from another apparatus. In this embodiment, the connection unit 16transmits a wide-view image acquired by the imaging processing unit 13to the information processing system 50. In some embodiments, thecommunication unit 11 may transmit the wide-view image to theinformation processing system 50.

The acceptance unit 12 is mainly implemented by the operation unit 115operating in accordance with instructions from the CPU 111. Theacceptance unit 12 is a function of accepting an operation input to theimage capturing apparatus 10 from the user. The acceptance unit 12accepts the operation of turning on or off the power, turning on or offa shutter button (start or stop of transmission of the wide-view image),an operation input to a touch panel, a button, or the like from theuser.

The imaging processing unit 13 is mainly implemented by the imageprocessor 104 operating in accordance with instructions from the CPU111. The imaging processing unit 13 captures an image of an object orsurroundings such as scenery and acquires (generates) a captured image.The captured image acquired by the imaging processing unit 13 may beeither or both of a moving image and a still image. In another example,the captured image may include an image and audio. Further, for example,the imaging processing unit 13 captures an image of a two-dimensionalcode (see FIG. 19B) displayed on the display 306 of the communicationterminal 30. The imaging processing unit 13 may execute the imageprocessing described with reference to FIGS. 7A to 8B on the capturedimage to generate a wide-view image.

The analysis unit 14 is mainly implemented in accordance withinstructions from the CPU 111. The analysis unit 14 analyzes thetwo-dimensional code, of which the image is captured by the imagingprocessing unit 13, to extract information included in thetwo-dimensional code. The extracted information includes a URL forregistering the image capturing apparatus 10 in the tenant, a temporaryID, and a password.

The registration request unit 15 is mainly implemented in accordancewith instructions from the CPU 111. The registration request unit 15transmits a request to the information processing system 50 via thecommunication unit 11 to register the image capturing apparatus 10 inthe tenant in the information processing system 50, by using theinformation included in the two-dimensional code read by the analysisunit 14.

The connection unit 16 is mainly implemented by the input/output I/F 116operating in accordance with instructions from the CPU 111. Theconnection unit 16 is a function of receiving a supply of power from thecommunication terminal 30A and performing data communication.

The storage processing unit 17 is mainly implemented in accordance withinstructions from the CPU 111. The storage processing unit 17 performs aprocess of storing a wide-view image captured in response to an imagecapturing request from any site in a URL (e.g., a URL that specifies thestorage 90) transmitted from the information processing system 50.

The image transmission control unit 18 is mainly implemented inaccordance with instructions from the CPU 111. The image transmissioncontrol unit 18 is a function of controlling transmission of thewide-view image to the information processing system 50. For example,the image transmission control unit 18 transmits a captured imageacquired by the imaging processing unit 13 to the information processingsystem 50 periodically or in response to a user operation when thecaptured image is a still image, or at a predetermined frame per second(FPS) when the captured image is a moving image. The image transmissioncontrol unit 18 also performs switching between the communication unit11 and the connection unit 16.

The adjustment unit 21 is mainly implemented in accordance withinstructions from the CPU 111. The adjustment unit 21 adjusts thequality of the wide-view image by using one or more image capturingparameters of the image capturing apparatus, which are related tovarious image capturing functions. Specifically, the quality of thewide-view image may be adjusted by adjusting, for example, exposure,white balance, International Organization for Standardization (ISO)sensitivity, shutter speed, focus, noise reduction, dynamic range (DR)correction, or high dynamic range (HDR) rendering, of the imagecapturing apparatus.

The storing/reading unit 19 is mainly implemented in accordance withinstructions from the CPU 111. The storing/reading unit 19 is a functionof storing various data in the storage unit 1000 or reading various datafrom the storage unit 1000. The storage unit 1000 stores captured imagedata acquired by the imaging processing unit 13, an image capturingapparatus ID, and the like. In an example, the captured image datastored in the storage unit 1000 may be deleted when a predeterminedamount of time has elapsed after the captured image data was acquired bythe imaging processing unit 13. In another example, the data transmittedto the information processing system 50 may be deleted.

The image capturing apparatus 10 has installed therein an application(also referred to as a plug-in) for supporting the communication system1 a. The application is used to associate the image capturing apparatus10 with the virtual room or to accept external control. Some of thefunctions illustrated in FIG. 13 , for example, the registration requestunit 15. are implemented by the application. The application forsupporting the communication system 1 a may reside on the communicationnetwork N, and a web browser or the like included in the image capturingapparatus 10 may access the application to implement similar functions.

Functional Configuration of Communication Terminal

Next, the functional configuration of the communication terminal 30 willbe described with reference to FIG. 13 . The communication terminal 30includes a communication unit 31, an acceptance unit 32, a displaycontrol unit 33. an imaging unit 34, a connection unit 36, astoring/reading unit 39. and a parameter generation unit 41. Each ofthese units is a function or means that is implemented by any one of thehardware elements illustrated in FIG. 4 operating in accordance withinstructions from the CPU 301 according to the program (either the webbrowser or a dedicated application) loaded onto the RAM 303. Thecommunication terminal 30 further includes a storage unit 3000. Thestorage unit 3000 is implemented by the ROM 302 or the recording medium315 illustrated in FIG. 4 .

The communication unit 31 is implemented by the network I/F 309operating in accordance with instructions from the CPU 301. Thecommunication unit 31 is a function of connecting to the communicationnetwork N and transmitting or receiving various data or information toor from any other device.

The acceptance unit 32 is mainly implemented by the keyboard 311 and thepointing device 312 operating in accordance with instructions from theCPU 301. The acceptance unit 32 is a function of accepting variousselection or operation inputs to the communication terminal 30. Thedisplay control unit 33 is a function of displaying a wide-view image,an image having a normal angle of view, and various screens on thedisplay 306 of the communication terminal 30.

The display control unit 33 is mainly implemented in accordance withinstructions from the CPU 301. For example, the display control unit 33causes the display 306 to display a two-dimensional code transmittedfrom the information processing system 50. In an example, thetwo-dimensional code is QR code®, DataMatrix (DataCode), MaxiCode, orPortable Document Format (PDF). In another example, the two-dimensionalcode is a barcode.

The imaging unit 34 is implemented by the camera 321 operating inaccordance with instructions from the CPU 301. The imaging unit 34captures an image of an object and surroundings.

The connection unit 36 is mainly implemented by the short-rangecommunication circuit 320 operating in accordance with instructions fromthe CPU 301. The connection unit 36 is a function of supplying power tothe image capturing apparatus 10 and performing data communication.

The parameter generation unit 41 is mainly implemented in accordancewith instructions from the CPU 301. The parameter generation unit 41determines whether the quality of a predetermined-area image displayedon the display 306 (an example of a display unit) is adequate, andgenerates image capturing parameters for making the quality of thepredetermined-area image adequate if the quality of thepredetermined-area image is not adequate. In an example, thecommunication terminal 30 does not include the parameter generation unit41 if the information processing system 50 includes a parametergeneration unit 61. For example, information regarding various imagecapturing functions representing the adequate image quality may bepreviously determined, and stored in any desired memory to be used fordetermination of whether the quality of the predetermined-area image isadequate. In this disclosure, in one example, the adequate quality maybe set such that an optimum image quality is achieved.

The storing/reading unit 39 is mainly implemented in accordance withinstructions from the CPU 301. The storing/reading unit 39 is a functionof storing various data in the storage unit 3000 or reading various datafrom the storage unit 3000. The storage unit 3000 includes an imagemanagement information storage unit 3001. The image managementinformation storage unit 3001 will be described in the description ofthe information processing system 50.

Functional Configuration of Information Processing System

Next, the functional configuration of the information processing system50 will be described. The information processing system 50 includes acommunication unit 51, a screen generation unit 52, an associationprocessing unit 53, an image distribution control unit 54, anauthentication unit 55, a communication group management unit 56, acommunication control unit 57, a connection management unit 58, astoring/reading unit 59, an API management unit 60, and a parametergeneration unit 61. Each of these units is a function or means that isimplemented by any one of the hardware elements illustrated in FIG. 4operating in accordance with instructions from the CPU 501 according tothe program loaded onto the RAM 503. The information processing system50 further includes a storage unit 5000. The storage unit 5000 isimplemented by the ROM 502, the HDD 504, or the recording medium 515illustrated in FIG. 4 .

The communication unit 51 is mainly implemented by the network I/F 509operating in accordance with instructions from the CPU 501. Thecommunication unit 51 is a function of transmitting or receiving variousdata or information to or from any other device via the communicationnetwork N.

The screen generation unit 52 is mainly implemented in accordance withinstructions from the CPU 501. The screen generation unit 52 generatesscreen information to be displayed on the communication terminal 30. Thescreen information is created by Hypertext Markup Language (HTML),Extensible Markup Language (XML), Cascade Style Sheet (CSS), JavaScript®(registered trademark), or any other language for a web application tobe executed by the communication terminal 30. For a native applicationto be executed by the communication terminal 30, the screen informationis held by the communication terminal 30, and the information to bedisplayed is transmitted in XML or the like. The screen generation unit52 generates screen information in which a wide-view image or the liketo be distributed by the image distribution control unit 54 through thecommunication unit 51 is arranged.

The association processing unit 53 is mainly implemented in accordancewith instructions from the CPU 501. The association processing unit 53performs control related to association and sharing of the point-of-viewinformation of the wide-view image. In response to receipt ofpoint-of-view information and an image capturing request from thecommunication terminal 30, the association processing unit 53 performs aprocess of associating the point-of-view information with a wide-viewimage acquired from the image capturing apparatus 10 in response to animage capturing request. The wide-view image and point-of-viewinformation, which are associated with each other, are stored in animage management information storage unit 5001 by the storing/readingunit 59. Further, the association processing unit 53 transmits storagelocation information (e.g., URLs) to the communication terminal 30 asinformation indicating storage locations where the associated wide-viewimage and point-of-view information are to be stored. In an example, theinformation processing system 50 does not simultaneously receive thepoint-of-view information and the image capturing request from thecommunication terminal 30. The information processing system 50separately receives the point-of-view information and the imagecapturing request and performs association processing. The URL is anexample of storage location information indicating a storage location.The storage location information may be in any other format such as auniform resource identifier (URI).

The image distribution control unit 54 is mainly implemented inaccordance with instructions from the CPU 501. The image distributioncontrol unit 54 distributes, to the communication terminal 30 operatedby a user who is in the virtual room, an image such as a wide-view imagetransmitted from the image capturing apparatus 10 associated with thesame virtual room, through the communication unit 51. An image having anormal angle of view captured by a camera included in the communicationterminal 30 or the camera 8 or 9 connected to the communication terminal30 is also distributed in a similar manner. The image to be distributedincludes streaming video, a moving image, and a still image.

The authentication unit 55 is mainly implemented in accordance withinstructions from the CPU 501. The authentication unit 55 is a functionof authenticating a request source in response to an authenticationrequest received by the communication unit 51. For example, theauthentication unit 55 determines whether authentication information (auser ID and a password) included in the authentication request receivedby the communication unit 51 matches authentication information held inadvance to perform user authentication. The authentication informationmay be the card number of an integrated circuit (IC) card, biometricauthentication information such as a face, a fingerprint, or avoiceprint, a device ID, a passcode, an access token, a security key, ora ticket. The authentication unit 55 may perform authentication using anexternal authentication system or an authentication method such as OpenAuthorization (Oauth). The authentication unit 55 may authenticate adevice such as an image capturing apparatus, instead of a user.

The communication group management unit 56 is mainly implemented inaccordance with instructions from the CPU 501. The communication groupmanagement unit 56 manages the entry of the communication terminal 30 orthe user into the virtual room, association between the virtual room anda device, and the like. Upon successful authentication of the user bythe authentication unit 55, the communication group management unit 56registers the user ID and the Internet protocol (IP) address of thecommunication terminal 30 in a virtual room information storage unit5002 or associates the image capturing apparatus 10 with the virtualroom.

The communication control unit 57 is mainly implemented in accordancewith instructions from the CPU 501. The communication control unit 57manages the start, establishment, and end of communication with theimage capturing apparatus 10 associated with each virtual room. Thecommunication control unit 57 also manages the start, establishment, andend of communication for distributing a wide-view image or audio inresponse to the communication terminal 30 entering or leaving thevirtual room.

The connection management unit 58 is mainly implemented in accordancewith instructions from the CPU 501. The connection management unit 58manages communication (connection) established with the informationprocessing system 50 by the communication terminal 30 and the imagecapturing apparatus 10 in association with the virtual room.

The API management unit 60 is mainly implemented in accordance withinstructions from the CPU 501. The API management unit 60 manages an APIto be used by a platform contractor to provide an image distributionservice of a wide-view image. In the use of the API. the platformcontractor develops software for calling the API. The software to bedeveloped may operate on a server or may operate on a client such as acommunication terminal. Any of the functions of the informationprocessing system 50, such as the image distribution control unit 54,the association processing unit 53, and the communication control unit57, can be provided as an API. Any function added to the informationprocessing system 50 later may be provided as an API. To determinewhether to provide a function as an API, a communication terminaloperated by the platform provider accesses the information processingsystem 50 and accepts the public settings of the API. As a result, theAPI management unit 60 can control the API based on the public settings.The API management unit 60 may perform an authentication process forchecking whether software operating on a request source that makes arequest to call the API is software developed by an authorized platformcontractor. The authentication process can be performed by comparinginformation registered and stored in advance in the storage unit 5000 asinformation on the platform contractor with information transmitted fromthe software operating on the request source.

In a specific example of the authentication process, the informationprocessing system 50 receives, from the software operating on therequest source, an application ID issued to the software developed bythe platform contractor in advance by the API management unit 60. If theAPI management unit 60 determines that the application ID matches anapplication ID stored in the storage unit 5000, the API management unit60 performs control to give permission to provide an API since thesoftware developed by the platform contractor is determined as beingvalid. If the software developed by the platform contractor is notdetermined as being valid, the API management unit 60 performs controlnot to give permission to provide an API.

The application ID is an example of authentication information fordetermining validity. The API management unit 60 may use authenticationinformation issued in advance by the API management unit 60 of theinformation processing system 50 or by an external system to check thevalidity of the request source. Examples of such authenticationinformation include an access token, a ticket, a security key, apassword, and a personal identification (PIN) code. In this embodiment,while the use of a function of the information processing system 50 asan API is not described, the same process flow is performed, except thatsoftware such as an application developed by a platform contractor usesa function of the information processing system 50 through adetermination made by the API management unit 60.

The parameter generation unit 61 is mainly implemented in accordancewith instructions from the CPU 501. The parameter generation unit 61determines whether the quality of a predetermined-area image displayedon the display 306 (an example of a display unit) of the communicationterminal 30 is adequate, and generates image capturing parameters formaking the quality of the predetermined-area image adequate if thequality of the predetermined-area image is not adequate. To recognize apredetermined area in the wide-view image, which is thepredetermined-area image displayed on the display 306 of thecommunication terminal 30, the parameter generation unit 61 acquires,from the communication terminal 30, point-of-view information(predetermined-area information) for identifying the predetermined areaas the predetermined-area image. In an example, the informationprocessing system 50 does not include the parameter generation unit 61if the communication terminal 30 includes the parameter generation unit41.

The storing/reading unit 59 is mainly implemented in accordance withinstructions from the CPU 501. The storing/reading unit 59 is a functionof storing various data in the storage unit 5000 or reading various datafrom the storage unit 5000.

Image Management Information Storage Unit 5001

The storage unit 5000 includes an image management information storageunit 5001. FIG. 14A is a table illustrating image management informationstored in the image management information storage unit 5001. The imagemanagement information storage unit 5001 stores image managementinformation as illustrated in FIG. 14A. The image management informationis information for managing wide-view images captured in response toimage capturing requests. In response to a user transmitting an imagecapturing request from the communication terminal 30, image managementinformation for one record is generated. The items contained in theimage management information will be described.

The item “data ID” of a wide-view image is identification informationthat identifies image data of the wide-view image. The informationprocessing system 50 numbers each data ID. ID is an abbreviation foridentification and means an identifier or identification information. IDis any one or a combination of two or more of a name, a symbol, acharacter string, and a numerical value that are used for uniquelyidentifying a specific object from among a plurality of objects. Eachdata ID may be associated with a wide-view image or an image captured ata normal angle of view by the image capturing apparatus 10 associatedwith the virtual room.

The item “data name” is the name of a wide-view image set by the user ofthe communication terminal 30. Each data name may be set by the user orautomatically.

The item “imaging date and time information” is information thatspecifies the imaging date and time of a captured image such as awide-view image. Examples of the imaging date and time include the dateand time when the user input an image capturing request to thecommunication terminal 30, and the date and time when the imagecapturing apparatus 10 captured an image such as a wide-view image. Theimaging date and time information may be time stamp information of acaptured image such as a wide-view image.

The item “imaging operator information” is identification information(including a user ID and a user name) of a user (imaging operator) whohas input an image capturing request to the communication terminal 30.Since a user inputs an image capturing request to the communicationterminal 30 after entering the virtual room, a user registered in theimaging operator information is identified by authentication to theinformation processing system 50 or the virtual room. The imagingoperator information is transmitted to the information processing system50 together with an image capturing request. In an example, an imagecapturing request and imaging operator information are not transmittedto the information processing system 50 at the same time, and aretransmitted to the information processing system 50 at differenttimings.

The item “image capturing apparatus information” is identificationinformation (image capturing apparatus ID) of the image capturingapparatus 10 that has captured an image to generate a wide-view image.The information processing system 50 numbers each image capturingapparatus ID and shares the image capturing apparatus ID with the imagecapturing apparatus 10. The image capturing apparatus ID may beinformation unique to the image capturing apparatus 10, such as a mediaaccess control (MAC) address or a serial number. The image capturingapparatus ID is transmitted to the information processing system 50together with the associated wide-view image. In an example, the imagecapturing apparatus ID and the associated wide-view image are nottransmitted to the information processing system 50 at the same time,and are transmitted to the information processing system 50 at differenttimings.

The item “imaging operator’s point-of-view information” is point-of-viewinformation designated for the communication terminal 30 of each imagingoperator. In an example, point-of-view information indicates thecoordinates of the center (center point illustrated in FIG. 11 ) of awide-view image currently displayed on the communication terminal 30.The point-of-view information is parameter information used foridentifying a predetermined area T (see FIGS. 9 and 10A to 10D) in awide-view image to be displayed on the communication terminal 30. In theillustrated example, the radius vector (r), the polar angle (θ), and theazimuth angle (φ) are given as the parameter information. Alternatively,any other parameter information described with reference to FIGS. 10A to12 may be used. The point-of-view information is transmitted from thecommunication terminal 30 that makes an image capturing request. Thepoint-of-view information may include information designating the widthand height of the display range of the predetermined area T.Alternatively, the point-of-view information may be the width and heightof the display range.

The item “imaging-time virtual room ID” is identification information ofa virtual room associated with the image capturing apparatus 10.

The item “data storage location information” (storage destinationinformation) of a wide-view image is information indicating a locationwhere the wide-view image is stored. The data storage locationinformation is represented by a URL, a file path, or the like. In anexample, the storage location identified by the storage locationinformation may be information indicating a predetermined folder. Thefolder may be a folder associated with a virtual room at an imagingtime. The folder may be a folder associated with identificationinformation (additional information such as a name) indicating any oneor a combination of two or more of categories such as the imaging dateand time, the image capturing apparatus, the imaging operator, and theimaging-time virtual room ID. In another example, the data storagelocation information may be used in combination with information such asa data ID and a data name to identify a data storage location.

FIG. 14B is a table illustrating image management information as amodification of the image management information illustrated in FIG.14A. In FIG. 14B, wide-view images having the same imaging-time virtualroom ID are stored. In this manner, image management information may beclassified in units of virtual rooms.

Virtual Room Information Storage Unit 5002

The storage unit 5000 includes the virtual room information storage unit5002. FIG. 15A is a table illustrating virtual room information storedin the virtual room information storage unit 5002. The virtual roominformation storage unit 5002 stores virtual room information asillustrated in FIG. 15A. The virtual room information is informationrelated to a virtual room. The virtual room information is held for eachvirtual room. The items contained in the virtual room information willbe described. In an example, the virtual room is registered in a tenant.In another example, the virtual room is not registered in a tenant, andinformation on a temporarily created virtual room or a virtual room tobe shared for use is also stored in the virtual room information storageunit 5002.

The item “virtual room ID” is identification information that identifiesthe virtual room. In this embodiment, each virtual room can be createdby a user as appropriate.

The item “virtual room name” is a name for the user to identify thevirtual room. Each virtual room name can be set by a user asappropriate. The virtual room ID and the virtual room name may be thesame information.

The item “device information” is identification information (device ID)of a device including the image capturing apparatus 10 associated withthe virtual room.

The item “user in virtual room” is the user ID of a user who has enteredand is currently in the virtual room. The user is a user authorized toview an image such as a wide-view image distributed to persons presentin the virtual room. The method for entering a virtual room will bedescribed below. The user ID may be associated with the IP address ofthe communication terminal 30 operated by the user. The user ID may bestored in association with a user name.

Tenant Information Storage Unit 5003

The storage unit 5000 includes a tenant information storage unit 5003.FIG. 15B is a table illustrating tenant information stored in the tenantinformation storage unit 5003. The tenant information storage unit 5003stores tenant information as illustrated in FIG. 15B. The tenantinformation is information related to a tenant (user group). The tenantinformation is held for each tenant. The items contained in the tenantinformation will be described. The tenant information includes variouskinds of information other than that illustrated in FIG. 15B, such asuser information. FIG. 15B illustrates part of the tenant information.

The item “tenant ID” is identification information that identifies thetenant.

The item “tenant name” is a name for the user to identify the tenant.The tenant ID and the tenant name may be the same information.

The item “tenant-registered virtual room ID” is identificationinformation of a virtual room registered in the tenant.

The item “tenant-registered device” is information related to a deviceregistered in the tenant.

The tenant information storage unit, the tenant ID, the tenant name, thetenant-registered virtual room ID, and the tenant-registered device mayalso be referred to as a user group information storage unit, a usergroup ID, a user group name, a user-group-registered virtual room ID,and a user-group-registered device, respectively.

Entry of Communication Terminal Into Virtual Room

Next, a process in which the user b enters the virtual room will bedescribed with reference to FIGS. 16A, 16B, and 17 . In the illustratedexample, the image capturing apparatus 10 has already been associatedwith a virtual room, and the communication terminal 30A has transmitteda wide-view image and an image having a normal angle of view to theinformation processing system 50. The association of the image capturingapparatus 10 with the virtual room and the like will be described withreference to FIG. 18 and the subsequent figures. In the followingdescription, no distinction is made between the entry of the user b intothe virtual room and the entry of the communication terminal 30B, whichis operated by the user b, into the virtual room.

FIGS. 16A and 16B illustrate examples of a screen displayed on thecommunication terminal 30B when the user b is to enter the virtual room.FIG. 16A illustrates an example of a room entry screen 200. Prior to thedisplay of the room entry screen 200, the user b logs into theinformation processing system 50. Upon the login of the user b, thetenant to which the user b belongs is identified. Virtual rooms areassociated with the tenant. A list of virtual rooms associated with thetenant is displayed on the communication terminal 30B (see FIG. 21 ),and the user b selects a virtual room that the user b is to enter fromthe list. FIG. 16A illustrates the room entry screen 200 for the virtualroom selected by the user b. A temporarily created virtual room or ashared virtual room, which is not associated with the tenant, may bedisplayed on the screen illustrated in FIG. 16A.

Alternatively, the creator of the virtual room may request theinformation processing system 50 to issue a URL corresponding to thevirtual room, and the URL may be transmitted to the user b via email orany other suitable means. In response to the user b clicking on the URLdisplayed on the communication terminal 30B. the communication terminal30B displays the room entry screen 200 illustrated in FIG. 16A.

The room entry screen 200 includes a virtual room name 201, aparticipant name input field 202, and a room entry button 203. Thevirtual room name 201 is the same as that stored in the virtual roominformation storage unit 5002. The participant name input field 202 is afield for entering a user name to be displayed in the virtual room. Theuser name may be a name such as a nickname of the user b. Upon the loginof the user b, a user name associated with the user ID of the user b maybe identified. The identified user name may be automatically displayed.The room entry button 203 is a button for the user b to send a requestto enter the virtual room.

At the time of entry into the virtual room, authentication for enteringthe virtual room may be requested separately from the login to thetenant.

FIG. 16B illustrates an image viewing screen 210 displayed on thecommunication terminal 30B upon the user b entering the virtual room.The image viewing screen 210 illustrated in FIG. 16B indicates that theimage capturing apparatus 10 has already started distributing awide-view image via the information processing system 50 and that thecommunication terminal 30A has already started distributing an imagehaving a normal angle of view. The image viewing screen 210 includes afirst image field 211 and a second image field 212. The first imagefield 211 displays the wide-view image, and the second image field 212displays the image having a normal angle of view. In an example, imagesare transmitted from three or more sites. The image viewing screen 210is divided into a number of portions corresponding to the number ofsites from which images are transmitted.

The first image field 211 displays a wide-view image mark 213. Thewide-view image mark 213 is set by the screen generation unit 52 of theinformation processing system 50 upon determination that the image to bedisplayed in the first image field 211 is a wide-view image. Thedetermination may be made by the communication terminal 30B, and thecommunication terminal 30B may display the wide-view image mark 213. Thewide-view image mark 213 allows the user b to know that a wide-viewimage for which the point of view can be changed has been distributed.The first image field 211 also displays a device name 214. The devicename 214 is transmitted from the image capturing apparatus 10 togetherwith the wide-view image. The device name 214 is information set by theuser a or the like (see FIGS. 19A and 19B), as described below.

The second image field 212 displays a participant name 215. Theparticipant name 215 is a user name. The participant name of a user whohas already entered the virtual room is displayed in the participantname input field 202. In the illustrated example, since the user a hasalready entered the virtual room, “AAA”, which is entered by the user ain the participant name input field 202, is displayed in the participantname input field 202.

FIG. 17 is a sequence diagram illustrating a process in which the user b(or the communication terminal 30B) enters the virtual room.

S1: First, the user b at the site B performs an operation of displayinga virtual room list screen. The communication terminal 30B accesses theinformation processing system 50 in advance in response to an operationperformed by the user b, and receives, from the information processingsystem 50, information on the virtual rooms stored in the virtual roominformation storage unit 5002 to display a virtual room list screen. Atthis time, the communication terminal 30B may transmit authenticationinformation used for login or the like to the information processingsystem 50 and may be authenticated by the authentication unit 55 of theinformation processing system 50. The authentication information may beauthentication information associated with the user b or authenticationinformation associated with the communication terminal 30B. The virtualrooms displayed in the virtual room list screen may be virtual roomsregistered in the tenant associated with the user b or virtual roomsregistered in the tenant associated with the communication terminal 30B.In response to the acceptance unit 32 accepting the operation ofdisplaying the virtual room list screen, the display control unit 33 ofthe communication terminal 30B causes the display 306 to display aselection screen.

S2: In response to the user b selecting a selection button for one ofthe virtual rooms, the acceptance unit 32 of the communication terminal30B accepts the selection of the virtual room. The display control unit33 of the communication terminal 30B causes the display 306 to displaythe room entry screen 200 illustrated in FIG. 16A.

S3: The user b completes the items and then presses the room entrybutton 203. In response to the acceptance unit 32 accepting the pressingof the room entry button 203, the communication unit 31 of thecommunication terminal 30B transmits a request to the informationprocessing system 50 to enter the virtual room. The request for enteringthe virtual room includes information such as a virtual room IDindicating the virtual room selected in step S2, the user ID of the userb authenticated by login or the like, and the IP address of thecommunication terminal 30B from which the request is transmitted. Then,the communication unit 51 of the information processing system 50receives the request for entering the virtual room.

S4: The communication group management unit 56 registers the IP addressand the user ID authenticated by login or the like in the virtual roominformation identified by the virtual room ID in the virtual roominformation storage unit 5002.

S5: The communication unit 51 of the information processing system 50transmits, to the communication terminal 30B, a response indicating thatthe communication terminal 30B has already entered the virtual room.Then, the communication unit 31 of the communication terminal 30Breceives the response indicating that the communication terminal 30B hasalready entered the virtual room. After step S5, the display controlunit 33 of the communication terminal 30B receives information on ascreen generated by the screen generation unit 52 of the informationprocessing system 50 and information on the image distributed by theimage distribution control unit 54, and causes the image viewing screen210 illustrated in FIG. 16B to be displayed based on the receivedinformation.

Association of Image Capturing Apparatus With Room

Next, an association of the image capturing apparatus 10 with a virtualroom will be described with reference to FIGS. 18 to 25 . In an example,the user a at the site A associates the image capturing apparatus 10with a virtual room. In another example, a system administrator, atenant administrator, or the like associates the image capturingapparatus 10 with a virtual room.

FIG. 18 illustrates an example of a device registration screen 220displayed on the communication terminal 30A. The user a has beenauthenticated by the information processing system 50 by login or thelike. Upon the login of the user a, the tenant to which the user abelongs is identified. The user a requests the information processingsystem 50 to display the device registration screen 220. Thecommunication terminal 30A displays the device registration screen 220received from the information processing system 50. First, a device isregistered in the tenant on the device registration screen 220.

The device registration screen 220 includes an image capturing apparatusregistration button 221, a VR goggles registration button 222, and asmart glasses registration button 223. Buttons are provided for therespective types of devices because the devices are different in thepresence or absence of a camera, information used for registration, orthe like. Since devices are registered on a per-type basis, theinformation processing system 50 can also grasp the types of thedevices.

The image capturing apparatus registration button 221 is a button forthe user a to register the image capturing apparatus 10. The VR gogglesregistration button 222 is a button for the user a to register the VRgoggles 89. The smart glasses registration button 223 is a button forthe user a to register the smart glasses 88.

FIGS. 19A and 19B illustrate an example of screens displayed in responseto the pressing of the image capturing apparatus registration button221. FIG. 19A illustrates an image capturing apparatus registrationdialog 230. The image capturing apparatus registration dialog 230includes a name field 231 for the image capturing apparatus 10, adescription field 232 for the image capturing apparatus 10, and a nextbutton 233. The user a sets any name in the name field 231 for the imagecapturing apparatus 10 and sets a description in the description field232 for the image capturing apparatus 10 such that the image capturingapparatus 10 to be registered can be identified.

In response to the user a pressing the next button 233, thecommunication terminal 30A requests a two-dimensional code from theinformation processing system 50. The communication terminal 30Adisplays the two-dimensional code.

FIG. 19B illustrates an example of a two-dimensional code screen 240displayed on the communication terminal 30A. The two-dimensional codescreen 240 includes a message 241 and a two-dimensional code 242. Themessage 241 indicates “Scan the two-dimensional code below to registerthe device named xxx (xxx is the name entered in the name field 231)”.The user a captures an image of the two-dimensional code 242 with theimage capturing apparatus 10 to be registered. The two-dimensional code242 includes a URL to which the image capturing apparatus 10 connects toregister the image capturing apparatus 10, and authenticationinformation used for registration, such as a temporary ID and apassword.

In response to the user a capturing the two-dimensional code 242 withthe image capturing apparatus 10, the image capturing apparatus 10connects to the URL and is authenticated using the temporary ID and thepassword. If the authentication is successful, the temporary ID isreplaced with a formal image capturing apparatus ID, and the name, thedescription, and the image capturing apparatus ID of the image capturingapparatus 10 are registered in the tenant. The image capturing apparatus10 also holds the image capturing apparatus ID, the name, and thedescription. The image capturing apparatus 10 registered in the tenantis associated with the virtual room in response to an operationperformed by the user a described below. The two-dimensional code 242 isan example of code information and has embedded therein similarauthentication information. In another example, the two-dimensional code242 is any other form of code such as a barcode.

Next, an example method for registering a communication terminal such asthe VR goggles 89 or the smart glasses 88 in a tenant will be describedwith reference to FIG. 20 . FIG. 20 illustrates an example of a VRgoggles registration screen 250 displayed in response to the pressing ofthe VR goggles registration button 222. The VR goggles registrationscreen 250 includes a temporary code input field 251 and a secret inputfield 252.

Without a camera, it is difficult for the VR goggles 89 to capture animage of a two-dimensional code. A temporary code (temporary ID) and asecret (password) may be output from (displayed on) the VR goggles 89,and the user a may enter the temporary code and the secret to thetemporary code input field 251 and the secret input field 252,respectively. The communication terminal 30A transmits the temporarycode and the secret to the information processing system 50 to registerthe VR goggles 89 in the tenant. The VR goggles 89 connect to theinformation processing system 50 and transmit the temporary code and thesecret. As a result, the VR goggles 89 are authenticated by theinformation processing system 50. If the authentication is successful,the temporary code is replaced with a formal VR goggles ID, and the VRgoggles ID is registered in the tenant. The VR goggles 89 also hold theVR goggles ID. The VR goggles 89 registered in the tenant are associatedwith a virtual room in response to an operation performed by the user adescribed below. The smart glasses 88 can also be registered by the usera in a manner similar to that for the image capturing apparatus 10 orthe VR goggles 89, which will be described in detail. The temporary codeand the secret are an example of authentication information. Otherinformation may be used as authentication information. The imagecapturing apparatus ID, the VR goggles ID, and the smart glasses ID areeach an example of a device ID and may also be referred to as deviceIDs. A device other than the image capturing apparatus 10, VR goggles,and smart glasses may be registered by using a device ID for associatingthe device with a virtual room or a tenant in a similar procedure. Thedevice ID may be identification information associated with the owner ofthe device.

FIG. 21 illustrates an example of a virtual room association screen 260for associating the image capturing apparatus 10 with a virtual room.The same screen configuration may be used for the VR goggles 89 and thesmart glasses 88. The virtual room association screen 260 includes avirtual room list 261. The virtual room list 261 displays individualvirtual room fields 262, 263, and 264. based on virtual rooms created inthe tenant. Each of the individual virtual room fields 262 to 264includes a link issuance button 265, a room entry button 266, a settingsbutton 267, and a virtual room name 268. The link issuance button 265 isa button for issuing a link (a URL for invitation) to the correspondingvirtual room and a passcode. The room entry button 266 is a button forthe user a to enter the virtual room. The settings button 267 is abutton for associating the image capturing apparatus 10 with the virtualroom. The virtual room name 268 is the same as that stored in thevirtual room information storage unit 5002. The user a presses thesettings button 267. In response to the pressing of the settings button267, the communication terminal 30A displays a virtual room associationscreen 270.

If a device has already been associated with the virtual room, a name269 of the device is displayed in the individual virtual room field (inFIG. 21 , the individual virtual room field 264).

FIG. 22 illustrates an example of the virtual room association screen270. The virtual room association screen 270 is displayed as a pop-up onthe virtual room association screen 260. In an example, the screentransition from the virtual room association screen 260 to the virtualroom association screen 270 is not made through the informationprocessing system 50. In another example, the screen transition from thevirtual room association screen 260 to the virtual room associationscreen 270 is made through the information processing system 50.

The virtual room association screen 270 includes a name 271 of the imagecapturing apparatus 10 that is currently (or has already been)associated with the virtual room, a connection button 272, a storagebutton 273, and a Save button 274. In FIG. 22 , the name 271 is setunregistered because the image capturing apparatus 10 is not registeredyet. The connection button 272 is a button for displaying a list ofdevices registered in the tenant as candidates to be associated with thevirtual room to associate a device with the virtual room. The storagebutton 273 is a button for displaying a list of storages 90 to store animage having a normal angle of view or a wide-view image captured by theimage capturing apparatus 10 associated with the virtual room. Examplesof the list of storages 90 may include a list of storages 90 to beassociated with the virtual room and a list of specific storagelocations such as folders in the storages 90. The user a selects apredetermined storage 90 or a specific storage location such as a folderin the storage 90 to associate the storage 90 with the virtual room.Information on the storage 90 associated with the virtual room in theway described above can be stored in the virtual room informationstorage unit 5002 in association with the virtual room ID. Examples ofthe information on the storage 90 include address information foraccessing the storage 90 and a storage location such as a folder in thestorage 90. In response to the pressing of the connection button 272,the communication terminal 30A displays a virtual room associationscreen 280.

The communication terminal 30A transmits a virtual room ID to theinformation processing system 50 and acquires the name (or ID) of adevice registered in the tenant for which the virtual room is generatedand the name (or ID) of a device associated with the virtual room.

FIG. 23 illustrates an example of the virtual room association screen280. The virtual room association screen 280 includes a name 281 of theimage capturing apparatus 10 that is currently (or has already been)associated with the virtual room, a list of devices 282 that can beadded, and a save button 283. The user a selects a device to beadditionally associated with the virtual room from the list of devices282 that can be added and then presses the save button 283. As a result,the selected device is associated with the virtual room. That is, adevice ID such as an image capturing apparatus ID is registered in thevirtual room information storage unit 5002. As illustrated in FIG. 23 ,the number of image capturing apparatuses to be associated with thevirtual room may be limited. In an example, the upper limit is two. Inthis case, the number of image capturing apparatus IDs that have alreadybeen registered in the virtual room information storage unit 5002 may bereferred to, and the remaining number of devices that can beadditionally registered may be displayed on the virtual room associationscreen 280.

Process for Starting Transmitting Wide-View Image to Image CapturingApparatus

In the way described above, a device such as the image capturingapparatus 10 is associated with the virtual room. The user a operatesthe device to start transmitting an image.

For the VR goggles 89 and the smart glasses 88, the user a operates thedevice main body to turn on or off the transmission of an image. This isbecause no application dedicated to the communication system 1 a iscurrently operating on the VR goggles 89 or the smart glasses 88. If anapplication dedicated to the communication system 1 a operates on the VRgoggles 89 and the smart glasses 88, the user a can also remotely turnon or off the transmission of an image.

For the image capturing apparatus 10, when the application is enabled,the user a can turn on or off the transmission of the wide-view imagefrom the menu of the application after entering the virtual room.

FIGS. 24A and 24B illustrate examples of a wide-view image transmissioncontrol dialog 290 displayed on the communication terminal 30A. Thewide-view image transmission control dialog 290 is displayed as a pop-upon the image viewing screen 210. In the illustrated example, the user ahas operated the communication terminal 30A and entered a virtual roomassociated with the image capturing apparatus 10. The wide-view imagetransmission control dialog 290 displays a name 292 of the imagecapturing apparatus 10 associated with the virtual room. A toggle button291 is displayed near the name 292. The user a can operate the togglebutton 291 to set the transmission of the wide-view image captured bythe image capturing apparatus 10 to on (start of transmission) or off(stop of transmission). The setting of on or off using a toggle buttonis an example. The start or stop of transmission of the wide-view imagemay be set in accordance with an input of a user operation. In anexample, the start or stop of transmission of the wide-view image may beset in response to selection of a radio button or a predetermined iconor in response to a menu operation. In another example, the transmissionof the wide-view image is started automatically, without the user’soperation, after the image capturing apparatus 10 enters the room. Inanother example, a predetermined condition such as the date and time,the number of users who have entered the room, or the participation of aspecific user is determined in advance, and the transmission of thewide-view image is started in response to a determination that thepredetermined condition is satisfied.

The wide-view image transmission control dialog 290 further displays asetting button 293 for setting various image capturing functions of theimage capturing apparatus 10. In response to pressing of the settingbutton 293, an image capturing function setting screen 400 illustratedin FIG. 27 is displayed. FIG. 27 is a view illustrating an example ofthe image capturing function setting screen 400. The image capturingfunction setting screen 400 displays a toggle button 401 for switchingbetween a manual setting mode and an automatic setting mode. In themanual setting mode, a user manually performs setting such as adjustmentfor exposure. In the automatic setting mode, the image capturingapparatus 10 automatically performs setting such as adjustment forexposure.

The automatic setting mode is a mode in which the image capturingparameters are transmitted to the image capturing apparatus 10 from anextraneous source, such as the display terminal or the informationprocessing system in this example, and the image capturing apparatus 10adjusts exposure and other settings based on the image capturingparameters. The automatic setting mode is a so-called forced automaticsetting mode. In the manual setting mode, the image capturing parametersare not transmitted to the image capturing apparatus 10 from theextraneous source. The image capturing apparatus 10 is directly operatedby a nearby user or a user makes a setting using a remote control toswitch between the automatic setting mode (“auto” mode) and the manualsetting mode (“manual” mode). At a time when the communication terminal30 switches between the automatic setting mode and the manual settingmode, the communication terminal 30 notifies the image capturingapparatus 10 of the automatic setting mode or the manual setting modevia the information processing system 50. The manual setting mode is setby default.

FIG. 27 illustrates an example of adjustment for exposure, forconvenience of illustration. Adjustment for white balance, ISOsensitivity, shutter speed, focus, noise reduction, DR correction, orHDR rendering is also available. In response to the user pressing a“Save” button 402 at the bottom right, the settings are stored. In thisembodiment, the following description will be given of a case where theautomatic setting mode is set.

The communication terminal 30A transmits transmission control settinginformation, which is set in response to an operation of the togglebutton 291, to the information processing system 50. The informationprocessing system 50 transmits a transmission start request or atransmission stop request corresponding to the transmission controlsetting information to the image capturing apparatus 10.

FIG. 24A illustrates the “off” setting of the toggle button 291. In FIG.24A, thus, the wide-view image is not displayed. By contrast, the imagehaving a normal angle of view captured by the camera 9 of thecommunication terminal 30A is displayed in the image viewing screen 210in FIG. 24A since the image having a normal angle of view has alreadybeen shared at the time of entry of the communication terminal 30A intothe virtual room.

FIG. 24B illustrates the “on” setting of the toggle button 291. Theinformation processing system 50 transmits a transmission start requestto the image capturing apparatus 10 in response to the toggle button 291being turned on, and the image capturing apparatus 10 startstransmitting the wide-view image. Since two images are shared in onevirtual room, the image viewing screen 210 is divided into two areas.Upon the setting being changed from the “on” setting to the “off”setting, the communication terminal 30A transmits “off” settinginformation to the information processing system 50. The informationprocessing system 50 transmits a transmission stop request to the imagecapturing apparatus 10 in response to receipt of the “off” settinginformation, and the image capturing apparatus 10 stops the transmissionof the wide-view image.

As described with reference to FIG. 25 , a user who is even on site canperform a simple operation such as capturing an image of codeinformation with the image capturing apparatus 10 to associate the imagecapturing apparatus 10 with the virtual room. In some cases, a user whois on site does not carry a PC or the like. It is useful that such auser who is on site can perform the association process on-site by usingthe image capturing apparatus 10 and code information issued in advance.In an example, if the user performs the association process in advance,the user can connect the image capturing apparatus 10 to a predeterminedvirtual room without selecting the virtual room or performing anyadditional operation. In addition, the user can give an instruction tostart or stop transmission from a remote site. This can reduce theburden on the user who desires to concentrate on work on-site.Accordingly, even in a preparation process, it is possible to provide asystem that enables efficient communication between on-site and remoteusers.

Procedure for Registering Image Capturing Apparatus in Virtual Room

Next, a procedure for registering the image capturing apparatus 10 inthe virtual room illustrated in the series of screen transitions inFIGS. 18 to 24B will be described with reference to FIG. 25 . FIG. 25 isa sequence diagram illustrating an example procedure in which the user aregisters the image capturing apparatus 10 in the virtual room.

S11: First, the user a connects the communication terminal 30A to theinformation processing system 50 and enters authentication information(such as a user ID and a password) to send a login request. Theacceptance unit 32 of the communication terminal 30A accepts theoperation.

S12: The communication unit 31 of the communication terminal 30Adesignates the authentication information and transmits the loginrequest to the information processing system 50. The communication unit51 of the information processing system 50 receives the login request,and the authentication unit 55 performs authentication based on thedesignated authentication information. It is assumed that theauthentication is successful. At this time, the information processingsystem 50 refers to the tenant information storage unit 5003 to identifythe tenant ID associated with the authenticated user ID.

S13: The screen generation unit 52 of the information processing system50 generates the device registration screen 220 in response to the useroperation, and the communication unit 51 transmits screen information ofthe device registration screen 220 to the communication terminal 30A.

S14: The communication unit 31 of the communication terminal 30Areceives the screen information of the device registration screen 220,and the display control unit 33 displays the device registration screen220 illustrated in FIG. 18 . The user a selects the type of the device(in the illustrated example, the image capturing apparatus 10 (e.g., aspherical camera)). Then, as illustrated in FIG. 19A, the user a entersthe name and description of the image capturing apparatus 10. Theacceptance unit 32 accepts the entered information.

S15: The communication unit 31 of the communication terminal 30Adesignates the name and description entered by the user a and transmitsa request for code information (e.g.. a two-dimensional code) to theinformation processing system 50.

S16: The communication unit 51 of the information processing system 50receives the request for code information (e.g., a two-dimensionalcode). The communication group management unit 56 generates a URL(connection destination for registration) in association with the nameand the description, and generates code information (e.g., atwo-dimensional code) including the URL, a temporary ID, and a password.The communication unit 51 of the information processing system 50transmits the code information (e.g., a two-dimensional code) to thecommunication terminal 30A. The communication unit 31 of thecommunication terminal 30A receives the code information (e.g., atwo-dimensional code). Then, the display control unit 33 displays thecode information (e.g., a two-dimensional code), as illustrated in FIG.19B.

S17: The user a operates the image capturing apparatus 10 to beassociated with the virtual room to capture an image of the codeinformation (e.g., a two-dimensional code). The acceptance unit 12 ofthe image capturing apparatus 10 accepts the operation.

S18: The imaging processing unit 13 of the image capturing apparatus 10performs an imaging process on an imaging target including the codeinformation (e.g., a two-dimensional code) to generate image data, andthe analysis unit 14 analyzes the image data to extract the URL, thetemporary ID, and the password. Accordingly, the registration requestunit 15 connects to the URL via the connection unit 16, designates thetemporary ID and the password, and transmits a request for registeringthe image capturing apparatus 10 to the information processing system50. In the registration method using the registration screen illustratedin FIG. 20 , an image of the code information is not captured. Thus, theimage capturing apparatus 10 is replaced with a communication terminalsuch as the VR goggles 89 or the smart glasses 88, and the steps S15 toS17 can be omitted.

S19: The communication unit 51 of the information processing system 50receives the temporary ID and the password, and the authentication unit55 determines whether the received temporary ID and password match thetemporary ID and password associated with the connected URL. It isassumed that a match is found.

S20: Since a request for registering the image capturing apparatus 10has been made, the communication group management unit 56 of theinformation processing system 50 generates an image capturing apparatusID as an example of a device ID and registers the image capturingapparatus ID in the tenant corresponding to the tenant ID identified atthe time of the login of the user a. The image capturing apparatus ID isassociated with a name and a description. Specifically, thecommunication group management unit 56 refers to the tenant informationstorage unit 5003 and additionally registers the image capturingapparatus ID in the tenant-registered device associated with theidentified tenant ID. In the illustrated example, the communicationgroup management unit 56 generates and registers an image capturingapparatus ID. In another example, the communication group managementunit 56 registers an image capturing apparatus ID received from theimage capturing apparatus 10. In the registration of, instead of theimage capturing apparatus 10, a communication terminal such as the VRgoggles 89 or the smart glasses 88 in the tenant, a device IDcorresponding to the communication terminal can be registered in thetenant information storage unit 5003 in a similar procedure.

S21: The communication unit 51 of the information processing system 50transmits the image capturing apparatus ID to the image capturingapparatus 10. The connection unit 16 of the image capturing apparatus 10receives the image capturing apparatus ID and stores the image capturingapparatus ID in the storage unit 1000.

S22: The communication terminal 30A is notified of the completion of theregistration by the communication unit 51 of the information processingsystem 50, and the user a can start associating the image capturingapparatus 10 with the virtual room in response to the notification. Theuser a selects, from the virtual room association screen 260 displayedon the communication terminal 30A, a virtual room with which the user adesires to associate the image capturing apparatus 10 registered in thetenant. The acceptance unit 32 of the communication terminal 30A acceptsan operation input indicating selection. Specifically, in response tothe acceptance unit 32 of the communication terminal 30A accepting anoperation input from the user a. the display control unit 33 causes thevirtual room association screen 260 to be displayed. At this time, thecommunication unit 31 may transmit a screen update request to thecommunication unit 51 of the information processing system 50. Inresponse to receipt of the screen update request, the informationprocessing system 50 refers to the tenant information storage unit 5003and identifies a virtual room ID registered in the tenant associatedwith the authenticated user ID. The information processing system 50further refers to the virtual room information storage unit 5002 andacquires the virtual room name associated with the identified virtualroom ID. The communication unit 51 of the information processing system50 transmits information on the identified virtual room ID and a virtualroom name corresponding to the identified virtual room ID (orinformation on a screen generated by the screen generation unit 52 basedon such information) to the communication terminal 30A. Thecommunication unit 31 of the communication terminal 30A receives theinformation on the virtual room ID and the virtual room name. Thedisplay control unit 33 can update and display the virtual roomassociation screen 260 in accordance with the received information. Suchinformation, which can be identified based on the user ID, may bereceived in step S13 after the authentication. The acceptance unit 32accepts an operation input indicating selection from the user a on thedisplayed virtual room association screen 260. As a result, thecommunication terminal 30A can identify the virtual room ID that hasbeen selected.

S23: The user a presses, on the virtual room association screen 270displayed on the communication terminal 30A, the connection button 272to additionally associate a device with the virtual room. The acceptanceunit 32 of the communication terminal 30A accepts an operation inputindicating the pressing of the connection button 272. Specifically, thedisplay control unit 33 of the communication terminal 30A displays thevirtual room association screen 270 corresponding to the selectedvirtual room ID identified in step S22. Further, the acceptance unit 32accepts, from the user a, an instruction to additionally associate adevice with the virtual room (the pressing of the connection button272).

S24: In response to the operation input in step S23, the communicationunit 31 of the communication terminal 30A requests the informationprocessing system 50 to provide information on devices registered in thetenant, which are candidate devices to be associated with the virtualroom, and devices that have already been associated with the virtualroom ID selected in step S22.

S25: The communication unit 51 of the information processing system 50receives the request for the information on the devices registered inthe tenant and the devices associated with the selected virtual room ID,and the screen generation unit 52 generates the virtual room associationscreen 280 including the device IDs of the devices registered in thetenant and the devices associated with the selected virtual room ID. Thecommunication unit 51 of the information processing system 50 transmitsscreen information of the virtual room association screen 280 to thecommunication terminal 30A.

S26: The communication unit 31 of the communication terminal 30Areceives the screen information of the virtual room association screen280, and the display control unit 33 causes the virtual room associationscreen 280 to be displayed. The user a selects a device to be associatedwith the virtual room. In an example, the user a selects the imagecapturing apparatus 10. The acceptance unit 32 of the communicationterminal 30A accepts the selection, and the image capturing apparatus IDis identified as the device ID of the selected device.

S27: The communication unit 31 of the communication terminal 30Adesignates the virtual room ID selected in step S22 and the device ID(e.g., the image capturing apparatus ID) selected in step S26, andtransmits an association request to the information processing system50.

S28: The communication unit 51 of the information processing system 50receives the association request, and the communication group managementunit 56 registers the device (e.g., the image capturing apparatus 10) inthe virtual room. That is, the communication group management unit 56refers to the virtual room information storage unit 5002 and registersthe device ID (e.g., the image capturing apparatus ID) in associationwith the virtual room ID designated in the request sent in step S27.

S29: Since the device ID (e.g., the image capturing apparatus ID) isassociated with the virtual room, the communication unit 51 of theinformation processing system 50 transmits the virtual room ID, thename, and the description to the image capturing apparatus 10. Theinformation processing system 50 may transmit the virtual room ID, thename, and the description to the image capturing apparatus 10 by using apush notification or by using polling, which is performed by the imagecapturing apparatus 10. The connection unit 16 of the image capturingapparatus 10 receives the virtual room ID, the name, and the descriptionand stores the virtual room ID, the name, and the description in thestorage unit 1000. Accordingly, the image capturing apparatus 10 can addthe image capturing apparatus ID, the virtual room ID, the name, thedescription, and the like to a wide-view image to be transmitted. Adevice other than the image capturing apparatus 10 can also beassociated with the virtual room in a similar procedure. Then, thecommunication unit 51 of the information processing system 50 maytransmit a notification indicating completion of the association to thecommunication terminal 30A. After step S29, the device (the imagecapturing apparatus 10) registered in association with the virtual roomcan connect to the associated virtual room. In the illustrated example,the image capturing apparatus 10 designates the virtual room ID receivedin step S29 and transmits a virtual room connection request to theinformation processing system 50, thereby connecting to the virtualroom. The timing at which the image capturing apparatus 10 connects tothe virtual room can be changed in accordance with an operationperformed by the user.

S30: The communication terminal 30A and the information processingsystem 50 perform the room entry process illustrated in FIG. 17 to allowthe communication terminal 30A to enter the virtual room associated withthe device (the image capturing apparatus 10).

S31: After entry into the virtual room, the user a turns on the togglebutton 291, on the image viewing screen 210, for the image capturingapparatus 10 associated with the virtual room. The acceptance unit 32 ofthe communication terminal 30A accepts the turn-on operation.

S32: The communication unit 31 of the communication terminal 30Adesignates the device ID (image capturing apparatus ID) and transmits,to the information processing system 50. a request for startingtransmission of the wide-view image. The user a may directly operate abutton of the image capturing apparatus 10 to start transmitting thewide-view image. In response to an operation performed by the user a,the communication unit 31 of the communication terminal 30A may transmita transmission stop request to the information processing system 50.

S33: The communication unit 51 of the information processing system 50receives the transmission start request and requests the image capturingapparatus 10 identified by the device ID (image capturing apparatus ID)to start transmission. The information processing system 50 may use apush notification or use polling, which is performed by the imagecapturing apparatus 10. The connection unit 16 of the image capturingapparatus 10 receives the transmission start request, and the imagingprocessing unit 13 starts capturing a wide-view image. The imagetransmission control unit 18 repeatedly transmits the wide-view imagewith a determined FPS or an FPS corresponding to a band via theconnection unit 16. As a result, the communication terminal 30 that hasentered the virtual room can display the state of the site A on theimage viewing screen 210 in real time.

Distribution of Wide-View Image and Others

A process for sharing a wide-view image or an image having a normalangle of view will be described with reference to FIG. 26 . FIG. 26 is asequence diagram illustrating an example process for sharing a wide-viewimage. In FIG. 26 , the communication terminals 30A and 30B have enteredthe virtual room by executing the room entry process described withreference to FIG. 17 . The communication terminal 30A includes thecamera 9 having a normal angle of view, and an image captured by thecamera 9 is shared with the communication terminal 30B. An imagecaptured by the smart glasses 88 associated with the virtual room,instead of the camera 9 of the communication terminal 30A, may beshared. In FIG. 26 , the image capturing apparatus 10 has already beenconnected to the same virtual room in accordance with the registrationprocedure illustrated in FIG. 25 .

S41: The imaging unit 34 of the communication terminal 30A captures animage, and the communication unit 31 designates the virtual room ID ofthe virtual room that the communication unit 31 is in and transmitsvideo including the captured image and audio to the informationprocessing system 50.

S42 and S43: In response to the communication unit 51 of the informationprocessing system 50 receiving the video including the image and theaudio, the image distribution control unit 54 acquires the IP addressesof the communication terminals 30A and 30B, which are in the samevirtual room, from the virtual room information storage unit 5002, andtransmits the video including the image and the audio via thecommunication unit 51. In FIG. 26 . an image having a normal angle ofview is received by the communication unit 31 of the communicationterminal 30A from the information processing system 50 and is displayed.In another example, an image having a normal angle of view is notreceived from the information processing system 50, but an image havinga normal angle of view is captured by the imaging unit 34 and isdisplayed.

S44: In response to a transmission start request based on the settingsfor startingtransmission, the imaging processing unit 13 of the imagecapturing apparatus 10 captures a wide-view image, and the imagetransmission control unit 18 designates the virtual room ID of thevirtual room in which the image capturing apparatus 10 is registered,the image capturing apparatus ID, and the name and the description ofthe image capturing apparatus 10 and transmits video including thewide-view image and audio to the information processing system 50 viathe connection unit 16.

S45 and S46: In response to the communication unit 51 of the informationprocessing system 50 receiving the video including the wide-view imageand the audio, the image distribution control unit 54 acquires the IPaddresses of the communication terminals 30A and 30B, which are in thesame virtual room, from the virtual room information storage unit 5002,and transmits the video including the wide-view image and the audio viathe communication unit 51.

S47: The communication terminal 30C including the camera 9 executes theroom entry process illustrated in FIG. 17 and newly enters the virtualroom.

S48: The communication unit 31 of the communication terminal 30Ctransmits video including an image having a normal angle of view andaudio to the information processing system 50.

S49 to S51: The communication unit 51 of the information processingsystem 50 receives the video including the image having a normal angleof view and the audio from the communication terminal 30C, and the imagedistribution control unit 54 acquires the IP addresses of thecommunication terminals 30A to 30C, which are in the same virtual room,from the virtual room information storage unit 5002, and transmits thevideo including the image having a normal angle of view and the audio.

S52: The communication unit 51 of the information processing system 50also transmits the video including the wide-view image and the audio tothe communication terminal 30C, which is in the same virtual room.

As described above, the users a and b, who are in the same virtual room,can share, in real time, the video including the wide-view imagecaptured by the image capturing apparatus 10 associated with the virtualroom. The order of transmission of the images illustrated in FIG. 26 isan example. In another example, the wide-view image may be shared first,or the image having a normal angle of view may be shared first.

A supplementary description will now be given of the smart glasses 88and the VR goggles 89. The smart glasses 88 have a camera having anormal angle of view and a display function. The camera of the smartglasses 88 captures an image having a normal angle of view, and thecaptured image having a normal angle of view is distributed in a mannersimilar to that for the cameras 8 and 9. The display function of thesmart glasses 88 is implemented by a flat screen, like that of anordinary display. Thus, part of the wide-view image is displayed from apoint of view designated by the user. The VR goggles 89 have a displayfunction. In an example, the VR goggles 89 may also include a camerahaving a normal angle of view. The display function of the VR goggles 89projects a wide-view image with a point of view determined by theorientation of the head of the user wearing the VR goggles 89. Thus, thepredetermined-area image Q of the predetermined area T in the wide-viewimage is displayed from a point of view corresponding to the orientationof the head of the user. While viewing a wide-view image with the smartglasses 88 or the VR goggles 89, the user can transmit, to theinformation processing system 50, an image capturing request thatdesignates point-of-view information of the wide-view image beingviewed.

Process for Automatically Adjusting Image Capturing Function

Next, a process for automatically adjusting the image capturing functionwill be described with reference to FIGS. 28 to 30 . The processillustrated in FIGS. 28 to 30 is performed when the automatic settingmode is set in FIG. 27 . FIG. 28 is a sequence diagram illustrating afirst process for adjusting the image capturing function. FIG. 29A is aview illustrating a wide-view image (spherical image). FIG. 29B is aview illustrating a predetermined-area image of a predetermined area T1.FIG. 29C is a view illustrating a predetermined-area image obtainedafter the image capturing function is adjusted.

First Process for Adjusting Image Capturing Function

S201: A user performs a screen operation to move the virtual point ofview while a predetermined-area image is displayed on the display 306 ofthe communication terminal 30 of the user. The predetermined-area imageis a predetermined area in a wide-view image distributed from the imagecapturing apparatus 10 via the information processing system 50. Theacceptance unit 32 accepts the movement of the point of view. Thedisplay control unit 33 causes the display 306 to display, for example,a predetermined-area image illustrated in FIG. 29B. Thepredetermined-area image is an image of the predetermined area T1 in thewide-view image illustrated in FIG. 29A. The predetermined-area imageillustrated in FIG. 29B is blurred, and the quality thereof is notadequate.

S202: The parameter generation unit 41 of the communication terminal 30determines whether the quality of the predetermined-area image beingdisplayed on the display 306 is adequate, and generates the imagecapturing parameters described above for making the quality of thepredetermined-area image adequate if the quality of thepredetermined-area image is not adequate. The case where the quality ofthe predetermined-area image is not adequate will be described.

S203: The communication unit 31 of the communication terminal 30transmits, to the information processing system 50, a request foradjusting the image capturing function of the image capturing apparatus10. The request includes the image capturing parameters generated instep S202. Then, the communication unit 51 of the information processingsystem 50 receives the request for adjusting the image capturingfunction.

S204: The communication unit 51 of the information processing system 50refers to the image management information stored in the imagemanagement information storage unit 5001 and transfers the request foradjusting the image capturing function to the image capturing apparatus10 located in the virtual room in which the communication terminal 30from which the request is received is also located. Then, the imagecapturing apparatus 10 receives the image capturing parameters and therequest for adjusting the image capturing function.

S205: The adjustment unit 21 of the image capturing apparatus 10 usesthe image capturing parameters received in step S204 to adjust thequality of the wide-view image that has been captured and is currentlybeing transmitted to the information processing system 50. In this case,the adjustment unit 21 notifies the imaging controller 105 of thecontent of the adjustment. The adjustment unit 21 performs adjustment tomake the quality of the predetermined-area image adequate. Morespecifically, the adjustment unit 21 performs adjustment on the entirewide-view image, not on the predetermined area in the wide-view image tobe transmitted. Since the adjustment unit 21 performs adjustment on thewide-view image to make the quality of the predetermined-area imageadequate, the image quality of an area other than the predetermined areain the wide-view image may degrade.

Accordingly, the wide-view image transmitted from the image capturingapparatus 10 after the adjustment of the quality of the wide-view imageis distributed to the communication terminal 30 via the informationprocessing system 50. In the communication terminal 30, the displaycontrol unit 33 can cause the display 306 to display apredetermined-area image whose quality is adequate. as illustrated inFIG. 29C.

Second Process for Adjusting Image Capturing Function

Next, a second process for adjusting the image capturing function. whichis a modification of the process illustrated in FIG. 28 ,will bedescribed with reference to FIG. 30 .

S301: As in step S201 described above, a user performs a screenoperation to move the virtual point of view while a predetermined-areaimage is displayed on the display 306 of the communication terminal 30of the user. The predetermined-area image is a predetermined area in awide-view image distributed from the image capturing apparatus 10 viathe information processing system 50. The acceptance unit 32 accepts themovement of the point of view. The display control unit 33 causes thedisplay 306 to display, for example, a predetermined-area imageillustrated in FIG. 29B. The predetermined-area image is an image of thepredetermined area T1 in the wide-view image illustrated in FIG. 29A.The predetermined-area image illustrated in FIG. 29B is blurred, and thequality thereof is not adequate.

S302: The communication unit 31 of the communication terminal 30transmits, to the information processing system 50, point-of-viewinformation for identifying a predetermined area that is a currentlydisplayed predetermined-area image. In this case, the point-of-viewinformation is displayed, regardless of whether the quality of thecurrently displayed predetermined-area image is adequate. Then, thecommunication unit 51 of the information processing system 50 receivesthe point-of-view information.

S303: In the information processing system 50, the parameter generationunit 61 determines whether the quality of the predetermined-area imageindicated by the point-of-view information received in step S302 isadequate in the currently distributed wide-view image, and generatesimage capturing parameters for making the quality of thepredetermined-area image adequate if the quality of thepredetermined-area image is not adequate.

S304 and S305: The processing of steps S304 and S305 is similar to thatof steps S204 and S205 described above, and the description thereof willthus be omitted.

Accordingly, the wide-view image transmitted from the image capturingapparatus 10 after the adjustment of the quality of the wide-view imageis distributed to the communication terminal 30 via the informationprocessing system 50. In the communication terminal 30, the displaycontrol unit 33 can cause the display 306 to display apredetermined-area image whose quality is adequate, as illustrated inFIG. 29C.

Application Example of Communication System in Telemedicine

FIG. 31 is a diagram illustrating an example of remote communication inwhich a communication system according to an embodiment of the presentdisclosure is used in telemedicine. FIG. 32 is a view illustrating anexample of a virtual room association screen for associating an imagecapturing apparatus with a virtual room for telemedicine.

A communication system 1 b according to an embodiment of the presentdisclosure will be described with reference to FIG. 31 , with focus ondifferences from FIG. 1 . In FIG. 31 , the site A is an operating room.The processing steps (1) to (6) in FIG. 31 are similar to those in FIG.1 . In FIG. 31 , a patient is placed on an operating table 355 and issubjected to surgery by a medical professional such as a surgeon. Themedical professional (corresponding to the user) operates on the patientwith various surgical tools 354 such as forceps and a scalpel. Themedical professional may wear the smart glasses 88 and transmit an imageof the surgical field for surgery performed by the medical professionalto the communication network N. Various cameras are placed in theoperating room. Examples of the cameras include a surgical camera 351, asurgical field camera 352, and an endoscope 353. All of the cameras inthe operating room and the smart glasses 88 are associated with avirtual room.

A main unit 356 is installed in the operating room. The main unit 356monitors the vitals of a patient, the operating state of medicaldevices, and the like. The main unit 356 corresponds to thecommunication terminal 30 according to this embodiment. Thecommunication terminal 30 (i.e., the main unit 356) in the operatingroom may have a function of receiving video from the endoscope 353 orthe surgical field camera 352 in addition to the functions illustratedin FIG. 1 . The communication terminal 30 can display the received videoon displays 306 and transmit the video to the information processingsystem 50 as the video of the site at which the communication terminal30 is located. An operation panel 357 is an input interface that acceptsvarious operations. In an example, a medical professional can operate adevice in the operating room via the operation panel 357. The endoscope353, the surgical camera 351, and the surgical field camera 352 maycommunicate with the information processing system 50 directly withoutthe intervention of the communication terminal 30.

The communication terminal 30 may have the function of an electronicmedical record system or the function of communicating with anelectronic medical record system. The communication terminal 30 maydisplay information on an electronic medical record on the display 306.

FIG. 32 is a view illustrating an example of a virtual room associationscreen 860 for associating an image capturing apparatus with a virtualroom for telemedicine. In the description of FIG. 32 , differences fromFIG. 21 will mainly be described.

In telemedicine, in an example, the virtual room association screen 860displays a list of virtual rooms 861 associated with remote surgery ormedical therapy. The site A is associated with a medical cameraincluding the image capturing apparatus 10, which is a spherical camera.Examples of the medical camera include an endoscope, a surgical fieldcamera for use in capturing a surgical field image in an operating room,and a camera that captures a microscopic image.

As described above, in one or more embodiments, when displaying apredetermined-area image that is a predetermined area in a wide-viewimage, a communication system identifies a predetermined area by usingpoint-of-view information to make the quality of the predetermined-areaimage adequate as much as possible.

The above-described embodiments are illustrative and do not limit thepresent disclosure. Thus, numerous additional modifications andvariations are possible in light of the above teachings. For example,elements and/or features of different illustrative embodiments may becombined with each other and/or substituted for each other within thescope of the present disclosure. Any one of the above-describedoperations may be performed in various other ways, for example, in anorder different from the one described above.

In the example configurations illustrated in FIG. 13 and the like, theinformation processing system 50, the image capturing apparatus 10, andthe communication terminal 30 are each divided according to mainfunctions to facilitate understanding of the processes performed by theinformation processing system 50, the image capturing apparatus 10, andthe communication terminal 30. No limitation is intended by how thefunctions are divided by process or by the name of the functions. Theprocessing of the information processing system 50, the image capturingapparatus 10, and the communication terminal 30 may be divided into moreunits of processing in accordance with the content of the processing.Further, the division may be made such that each unit of processingincludes more processing operations.

Any one of the above-described operations may be performed in variousother ways, for example, in an order different from the one describedabove.

Each of the functions of the described embodiments may be implemented byone or more processing circuits or circuitry. As used herein, the term“processing circuit or circuitry” includes a processor programmed toimplement each function by software, such as a processor implemented byan electronic circuit, and devices designed to implement the functionsdescribed above, such as an application specific integrated circuit(ASIC), a digital signal processor (DSP), a field programmable gatearray (FPGA), and existing circuit modules.

The apparatuses or devices described in one or more embodiments are justone example of plural computing environments that implement the one ormore embodiments disclosed herein. In some embodiments, the informationprocessing system 50 includes multiple computing devices, such as aserver cluster. The multiple computing devices are configured tocommunicate with one another through any type of communication link,including a network, a shared memory, etc., and perform processesdisclosed herein.

Further, the information processing system 50 can be configured to sharethe processing steps disclosed herein, for example, the processing stepsillustrated in FIGS. 25 and 26 and so on in various combinations. Forexample, a process executed by a predetermined unit may be executed by aplurality of information processing apparatuses included in theinformation processing system 50. The components of the informationprocessing system 50 may be integrated into one server apparatus ordivided into a plurality of apparatuses.

According to Aspect 1, a display terminal is provided. The displayterminal acquires a wide-view image captured by an image capturingapparatus and having a viewing angle in a wide range, and displays apredetermined-area image representing a predetermined area of thewide-view image. The display terminal includes: a display control unitthat causes a display to display the predetermined-area image; and aparameter transmitting unit that transmits an image capturing parameterfor making quality of the predetermined-area image being displayed onthe display adequate, such that the image capturing parameter isreceivable by the image capturing apparatus.

According to Aspect 2, in the display terminal according to aspect 1,the image capturing parameter includes a parameter for adjustment forexposure, white balance, International Standards Organization (ISO)sensitivity, shutter speed, focus, noise reduction, dynamic range (DR)correction, or high dynamic range (HDR) rendering of the image capturingapparatus.

According to Aspect 3, the display terminal according to aspect 1 or 2.further includes: a parameter generation unit that determines whetherthe quality of the predetermined-area image being displayed on thedisplay is adequate and generates the image capturing parameter based ona determination that the quality of the predetermined-area image beingdisplayed on the display is not adequate. The parameter transmittingunit transmits the generated image capturing parameter.

According to Aspect 4, a display method is performed by a displayterminal that acquires a wide-view image captured by an image capturingapparatus and having a viewing angle in a wide range, and displays apredetermined-area image representing a predetermined area of thewide-view image. The display method includes, by the display terminal:causing a display to display the predetermined-area image; andtransmitting an image capturing parameter for making quality of thepredetermined-area image being displayed on the display adequate, suchthat the image capturing parameter is receivable by the image capturingapparatus.

According to Aspect 5, a non-transitory recording medium storing aplurality of instructions which, when executed by one or moreprocessors, causes the processors to perform a display method with adisplay terminal is provided. The display terminal acquires a wide-viewimage captured by an image capturing apparatus and having a viewingangle in a wide range, and displays a predetermined-area imagerepresenting a predetermined area of the wide-view image. The displaymethod includes, by the display terminal: causing a display to displaythe predetermined-area image; and transmitting an image capturingparameter for making quality of the predetermined-area image beingdisplayed on the display adequate, such that the image capturingparameter is receivable by the image capturing apparatus.

According to Aspect 6, an image capturing apparatus is provided, whichcaptures a wide-view image having a viewing angle in a wide range andtransmits the captured wide-view image. The image capturing apparatusincludes: an information receiving unit that receives an image capturingparameter transmitted from a display terminal that displays apredetermined-area image on a display, the predetermined-area imageindicating a predetermined area of the transmitted wide-view image, theimage capturing parameter being for making quality of thepredetermined-area image being displayed on the display adequate; anadjustment unit that adjusts quality of the wide-view image by using theimage capturing parameter; and an image transmitting unit that transmitsthe wide-view image whose quality is adjusted by the adjustment unit.

According to Aspect 7, in the image capturing apparatus according toaspect 6, the image capturing parameter includes a parameter foradjustment for exposure, white balance, International StandardsOrganization (ISO) sensitivity, shutter speed, focus, noise reduction,dynamic range (DR) correction, or high dynamic range (HDR) rendering, ofthe image capturing apparatus.

According to Aspect 8, in the image capturing apparatus according toaspect 6 or 7, the wide-view image includes a spherical image inequirectangular projection format, an omnidirectional image, ahemispherical image, a three-dimensional panoramic image, atwo-dimensional panoramic image, or a virtual reality (VR) image.

According to Aspect 9, an image capturing method performed by an imagecapturing apparatus is provided. The image capturing apparatus capturesa wide-view image having a viewing angle in a wide range and transmitsthe captured wide-view image. The image capturing method includes, bythe image capturing apparatus: receiving an image capturing parametertransmitted from a display terminal that displays a predetermined-areaimage on a display, the predetermined-area image representing apredetermined area of the transmitted wide-view image, the imagecapturing parameter being for making quality of the predetermined-areaimage being displayed on the display adequate; adjusting quality of thewide-view image by using the image capturing parameter: and transmittingthe wide-view image whose quality is adjusted.

According to Aspect 10, a non-transitory recording medium is provided,which stores a plurality of instructions which, when executed by one ormore processors, causes the processors to perform an image capturingmethod with an image capturing apparatus. The image capturing apparatuscaptures a wide-view image having a viewing angle in a wide range andtransmits the captured wide-view image. The image capturing methodincludes, by the image capturing apparatus: receiving an image capturingparameter transmitted from a display terminal that displays apredetermined-area image on a display, the predetermined-area imagerepresenting a predetermined area in the transmitted wide-view image,the image capturing parameter being for making quality of thepredetermined-area image being displayed on the display adequate;adjusting quality of the wide-view image by using the image capturingparameter; and transmitting the wide-view image whose quality isadjusted.

According to Aspect 11, a communication system includes: an imagecapturing apparatus that captures a wide-view image having a viewingangle in a wide range and transmits the captured wide-view image; and adisplay terminal that acquires the wide-view image transmitted from theimage capturing apparatus and displays a predetermined-area imagerepresenting a predetermined area of the wide-view image. The displayterminal includes: a display control unit that causes a display todisplay the predetermined-area image; and a parameter transmitting unitthat transmits an image capturing parameter for making quality of thepredetermined-area image being displayed on the display adequate, suchthat the image capturing parameter is receivable by the image capturingapparatus. The image capturing apparatus includes: an adjustment unitthat adjusts quality of the wide-view image by using the received imagecapturing parameter; and an image transmitting unit that transmits thewide-view image whose quality is adjusted by the adjustment unit.

According to Aspect 12, in the communication system according to aspect11, wherein the image capturing parameter includes a parameter foradjustment for exposure, white balance, International StandardsOrganization (ISO) sensitivity, shutter speed, focus, noise reduction,dynamic range (DR) correction, or high dynamic range (HDR) rendering of,the image capturing apparatus.

According to Aspect 13, in the communication system according to aspect11 or 12, the display terminal further includes a parameter generationunit that determines whether the quality of the predetermined-area imagebeing displayed on the display is adequate and generates the imagecapturing parameter based on a determination that the quality of thepredetermined-area image being displayed on the display is not adequate,and the parameter transmitting unit that transmits the generated imagecapturing parameter.

According to Aspect 14, the communication system according to aspect 11or 12, further includes: an information processing system (may bereferred to as the information processing server) that relayscommunication between the image capturing apparatus and the displayterminal

According to Aspect 15, a communication method performed by acommunication system is provided. The communication system includes: animage capturing apparatus that captures a wide-view image having aviewing angle in a wide range and transmits the captured wide-viewimage; and a display terminal that acquires the wide-view imagetransmitted from the image capturing apparatus and displays apredetermined-area image representing a predetermined area of thewide-view image. The communication method includes: causing, by thedisplay terminal, a display to display the predetermined-area image:transmitting, by the display terminal, an image capturing parameter formaking quality of the predetermined-area image being displayed on thedisplay adequate, such that the image capturing parameter is receivableby the image capturing apparatus; receiving, by the image capturingapparatus, the image capturing parameter; adjusting, by the imagecapturing apparatus, quality of the wide-view image by using the imagecapturing parameter; and transmitting, by the image capturing apparatus,the wide-view image whose quality is adj usted.

According to Aspect 16, an information processing system receives awide-view image captured by an image capturing apparatus and having aviewing angle in a wide range, and distributes the wide-view image to adisplay terminal. The information processing system includes: aninformation receiving unit that receives point-of-view informationtransmitted from the display terminal, the point-of-view informationindicating a predetermined area of the wide-view image being displayedon a display: and a parameter transmitting unit that transmits, to theimage capturing apparatus, an image capturing parameter for makingquality of the predetermined-area image adequate, representing thepredetermined area of the wide-view image indicated by the point-of-viewinformation.

According to Aspect 17, the information processing system according toaspect 16, further includes: a parameter generation unit that determineswhether the quality of the predetermined-area image represented by thepredetermined area of the wide-view image, indicated by thepoint-of-view information, is adequate, and generates the imagecapturing parameter based on a determination that the quality of thepredetermined-area image is not adequate. The parameter transmittingunit transmits the generated image capturing parameter.

According to Aspect 18, an information processing method performed by aninformation processing system is provided. The information processingsystem receives a wide-view image captured by an image capturingapparatus and having a viewing angle in a wide range, and distributesthe wide-view image to a display terminal. The information processingmethod includes, by the information processing system: receivingpoint-of-view information transmitted from the display terminal, thepoint-of-view information indicating a predetermined area of thewide-view image being displayed on a display; and transmitting, to theimage capturing apparatus, an image capturing parameter for makingquality of the predetermined-area image adequate, represented by thepredetermined area of the wide-view image, indicated by thepoint-of-view information.

According to Aspect 19, a non-transitory recording medium is provided,which stores a plurality of instructions which, when executed by one ormore processors, causes the processors to perform an informationprocessing method with an information processing system. The informationprocessing system receives a wide-view image captured by an imagecapturing apparatus and having a viewing angle in a wide range, anddistributes the wide-view image to a display terminal. The informationprocessing method includes, by the information processing system:receiving point-of-view information transmitted from the displayterminal, the point-of-view information indicating a predetermined areaof the wide-view image being displayed on a display: and transmitting,to the image capturing apparatus, an image capturing parameter formaking quality of the predetermined-area image represented by thepredetermined area of the wide-view image adequate, indicated by thepoint-of-view information.

According to Aspect 20, an image capturing apparatus captures awide-view image having a viewing angle in a wide range and transmits thecaptured wide-view image, includes: an information receiving unit thatreceives an image capturing parameter transmitted from an informationprocessing system. The image capturing parameter is for making qualityof a predetermined-area image adequate, represented by a predeterminedarea of the wide-view image indicated by point-of-view information,based on the point-of-view information that is transmitted from adisplay terminal that displays the predetermined-area image on adisplay. The predetermined-area image represents the predetermined areaof the wide-view image. The point-of-view information indicates thepredetermined area of the wide-view image, which is displayed as thepredetermined-area image on the display. The image capturing apparatusfurther includes: an adjustment unit that adjusts quality of thewide-view image by using the image capturing parameter; and an imagetransmitting unit that transmits the wide-view image whose quality isadjusted by the adjustment unit.

According to Aspect 21, an image capturing method performed by an imagecapturing apparatus that captures a wide-view image having a viewingangle in a wide range and transmits the captured wide-view image, isprovided. The image capturing method includes, by the image capturingapparatus: receiving an image capturing parameter transmitted from aninformation processing system, the image capturing parameter being formaking quality of a predetermined-area image representing apredetermined area of the wide-view image adequate, indicated bypoint-of-view information, based on the point-of-view information thatis transmitted from a display terminal that displays thepredetermined-area image on a display. The predetermined-area imagerepresents the predetermined area of the wide-view image. Thepoint-of-view information indicates the predetermined area of thewide-view image, which is the predetermined-area image being displayedon the display. The image capturing method further includes: adjustingquality of the wide-view image by using the image capturing parameter;and transmitting the wide-view image whose quality is adj usted.

According to Aspect 22, a non-transitory recording medium stores aplurality of instructions which, when executed by one or moreprocessors, causes the processors to perform an image capturing methodwith an image capturing apparatus. The image capturing apparatuscaptures a wide-view image having a viewing angle in a wide range andtransmits the captured wide-view image. The image capturing methodincludes, by the image capturing apparatus: receiving an image capturingparameter transmitted from an information processing system, the imagecapturing parameter being for making quality of a predetermined-areaimage representing a predetermined area of the wide-view image adequate,indicated by point-of-view information, based on the point-of-viewinformation that is transmitted from a display terminal that displaysthe predetermined-area image on a display. The predetermined-area imagerepresents the predetermined area of the wide-view image. Thepoint-of-view information indicates the predetermined area of thewide-view image, which is the predetermined-area image being displayedon the display. The image capturing method further includes: adjustingquality of the wide-view image by using the image capturing parameter;and transmitting the wide-view image whose quality is adjusted.

According to Aspect 23, a communication system includes: an imagecapturing apparatus that captures a wide-view image having a viewingangle in a wide range and transmits the captured wide-view image: aninformation processing system that receives the wide-view imagetransmitted from the image capturing apparatus and distributes thewide-view image to a display terminal; and the display terminal thatdisplays a predetermined-area image representing a predetermined area ofthe wide-view image distributed by the information processing system.The display terminal includes: a display control unit that causes adisplay to display the predetermined-area image; and an informationtransmitting unit that transmits, to the information processing system,point-of-view information for indicating the predetermined area of thewide-view image, represented by the predetermined-area image beingdisplayed on the display. The information processing system includes: aparameter transmitting unit that transmits, to the image capturingapparatus, an image capturing parameter for making quality of thepredetermined-area image representing the predetermined area of thewide-view image adequate, the predetermined area being indicated by thepoint-of-view information. The image capturing apparatus includes: anadjustment unit that adjusts quality of the wide-view image by using thereceived image capturing parameter; and an image transmitting unit thattransmits, to the information processing system, the wide-view imagewhose quality is adjusted by the adjustment unit.

According to Aspect 24, a communication method performed by acommunication system is provided. The communication system includes: animage capturing apparatus that captures a wide-view image having aviewing angle in a wide range and transmits the captured wide-viewimage; an information processing system that receives the wide-viewimage transmitted from the image capturing apparatus and distributes thewide-view image to a display terminal; and the display terminal thatdisplays a predetermined-area image representing a predetermined area ofthe wide-view image distributed by the information processing system.The communication method includes: causing, by the display terminal, adisplay to display the predetermined-area image; transmitting, by thedisplay terminal, to the information processing system, point-of-viewinformation indicating the predetermined area of the wide-view image,represented by the predetermined-area image being displayed on thedisplay; distributing, by the information processing system, to theimage capturing apparatus, an image capturing parameter for makingquality of the predetermined-area image adequate, the predetermined-areaimage being indicated by the point-of-view information of the wide-viewimage being distributed; adjusting, by the image capturing apparatus,quality of the wide-view image by using the received image capturingparameter; and transmitting, by the image capturing apparatus, to theinformation processing system, the wide-view image whose quality isadjusted.

1. A display terminal comprising circuitry configured to: acquire awide-view image captured by an image capturing apparatus and having awide angle of view; display, on a display, a predetermined-area imagerepresenting a predetermined area of the wide-view image; and transmitan image capturing parameter to be received by the image capturingapparatus, the image capturing parameter being determined such that thequality of the predetermined-area image being displayed is adequate. 2.The display terminal according to claim 1, wherein the image capturingparameter adjusts at least one of exposure, white balance, InternationalStandards Organization (ISO) sensitivity, shutter speed, focus, noisereduction, dynamic range (DR) correction, or high dynamic range (HDR)rendering, of the image capturing apparatus.
 3. The display terminalaccording to claim 1, wherein the circuitry is configured to: determinewhether the quality of the predetermined-area image being displayed isadequate; generate the image capturing parameter for making the qualityof the predetermined-area image adequate based on a determination thatthe quality of the predetermined-area image being displayed is notadequate; and transmit the generated image capturing parameter.
 4. Asystem comprising: the display terminal according to claim 1; and animage capturing apparatus configured to capture the wide-view image fordisplay at the display terminal, the image capturing apparatus includingcircuitry configured to: receive the image capturing parameter; adjustquality of the wide-view image using the image capturing parameter; andtransmit the wide-view image having the adjusted quality for display atthe display terminal.
 5. The system of claim 4, further comprising: aninformation processing server configured to relay communication betweenthe image capturing apparatus and the display terminal.
 6. The system ofclaim 5, wherein the information processing server includes circuitryconfigured to: receive a request for adjusting an image capturingfunction of the image capturing apparatus from the display terminal, therequest including the image capturing parameter; and transmit therequest for adjusting to the image capturing apparatus.
 7. A displayingmethod comprising: acquiring a wide-view image captured by an imagecapturing apparatus and having a wide angle of view; displaying, on adisplay, a predetermined-area image representing a predetermined area ofthe wide-view image; and transmitting an image capturing parameter to bereceived by the image capturing apparatus, the image capturing parameterbeing determined such that the quality of the predetermined-area imagebeing displayed is adequate.
 8. A non-transitory recording mediumstoring a plurality of instructions which, when executed by one or moreprocessors, causes the processors to perform a display methodcomprising: acquiring a wide-view image captured by an image capturingapparatus and having a wide angle of view; displaying, on a display, apredetermined-area image representing a predetermined area of thewide-view image; and transmitting an image capturing parameter to bereceived by the image capturing apparatus, the image capturing parameterbeing determined such that the quality of the predetermined-area imagebeing displayed is adequate.